Image capturing device, image processing device and display device for setting different exposure conditions

ABSTRACT

An image capturing device includes: an image capturing element having a first image capturing region that captures an image of a photographic subject and outputs a first signal, and a second image capturing region that captures an image of the photographic subject and outputs a second signal; a setting unit that sets an image capture condition for the first image capturing region to an image capture condition that is different from an image capture condition for the second image capturing region; a correction unit that performs correction upon the second signal, for employment in interpolation of the first signal; and a generation unit that generates an image of the photographic subject that has been captured by the first image capturing region by employing a signal generated by interpolating the first signal according to the second signal as corrected by the correction unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Divisional of application Ser. No. 17/027,161, filed Sep. 21,2020, which in turn is a Continuation of application Ser. No.15/765,129, filed Mar. 30, 2018 (now U.S. Pat. No. 10,810,716), which isa National Phase Application of PCT/JP2016/078280, filed Sep. 26, 2016,which claims the benefit of Japanese Patent Application No. 2015-194612,filed Sep. 30, 2015. The disclosure of each of the prior applications ishereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

The present invention relates to an image capturing device, to an imageprocessing device, and to a display device.

BACKGROUND ART

An image capturing device is per se known (refer to Patent Document #1)that is equipped with an imaging element that is capable of settingdifferent image capture conditions for various regions of the screen.However, when processing captured image data that has been generated forseveral regions whose image capture conditions are different, noconsideration has been accorded to the image capture conditions.

CITATION LIST Patent Literature

Patent Document #1: Japanese Laid-Open Patent Publication 2006-197192.

SUMMARY OF INVENTION

According to a first aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatperforms correction upon the second signal outputted from the secondimage capturing region, for employment in interpolation of the firstsignal outputted from the first image capturing region; and a generationunit that generates an image of the photographic subject that has beencaptured by the first image capturing region by employing a signalgenerated by interpolating the first signal according to the secondsignal as corrected by the correction unit.

According to a second aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatperforms correction upon the second signal outputted from the secondimage capturing region, for employment in interpolation of a pixel inthe first image capturing region that outputs the first signal; and ageneration unit that generates an image of the photographic subject thathas been captured by the first image capturing region by employing asignal generated by interpolating the pixel that outputs the firstsignal according to the second signal corrected by the correction unit.

According to a third aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatperforms correction upon the second signal outputted from the secondimage capturing region, for reducing noise included in the first signaloutputted from the first image capturing region; and a generation unitthat reduces noise included in the first signal by employing the secondsignal corrected by the correction unit, and generates an image of thephotographic subject that has been captured by the first image capturingregion.

According to a fourth aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatperforms correction upon the second signal outputted from the secondimage capturing region, for employment in image processing of the firstsignal outputted from the first image capturing region; and a generationunit that performs the image processing of the first signal by employingthe second signal corrected by the correction unit, and generates animage of the photographic subject that has been captured by the firstimage capturing region.

According to a fifth aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region andan image capture condition for the second image capturing region; aprocessing unit that, as processing to interpolate the first signal byemploying the second signal outputted from the second image capturingregion, performs different processing depending upon the image capturecondition for the second image capturing region set by the setting unit;and a generation unit that generates an image of the photographicsubject that has been captured by the first image capturing region, byemploying a signal generated by interpolating the first signal by theprocessing unit.

According to a sixth aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal, a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatcorrects the first signal outputted from the first image capturingregion, and performs correction upon the second signal outputted fromthe second image capturing region, for employment in interpolation ofthe corrected first signal; and a generation unit that generates animage of the photographic subject that has been captured by the firstimage capturing region by employing a signal generated by interpolatingthe first signal corrected by the correction unit according to thesecond signal corrected by the correction unit.

According to a seventh aspect, an image capturing device comprises: animage capturing element having a first image capturing region thatcaptures an image of a photographic subject and outputs a first signal,and a second image capturing region that captures an image of thephotographic subject and outputs a second signal; a setting unit thatsets an image capture condition for the first image capturing region toan image capture condition that is different from an image capturecondition for the second image capturing region; a correction unit thatcorrects the first signal outputted from the first image capturingregion, and performs correction upon the second signal outputted fromthe second image capturing region, for employment in interpolation of apixel that outputs the corrected first signal, and a generation unitthat generates an image of the photographic subject that has beencaptured by the first image capturing region by employing a signalgenerated by interpolating the pixel that outputs the first signalcorrected according to the second signal corrected by the correctionunit.

According to an eighth aspect, an image processing device comprises: acorrection unit that, for employment in interpolation of a first signaloutputted from a first image capturing region of an image capturingelement that captures an image of a photographic subject, performscorrection upon a second signal outputted from a second image capturingregion of the image capturing element for which an image capturecondition is different from an image capture condition for the firstimage capturing region; and a generation unit that generates an image ofthe photographic subject that has been captured by the first imagecapturing region by employing a signal generated by interpolating thefirst signal according to the second signal corrected by the correctionunit.

According to a ninth aspect, an image processing device comprises: acorrection unit that, in order to reduce noise included in a firstsignal outputted from a first image capturing region of an imagecapturing element that captures an image of a photographic subject,performs correction upon a second signal outputted from a second imagecapturing region of the image capturing element for which an imagecapture condition is different from an image capture condition for thefirst image capturing region; and a generation unit that reduces noiseincluded in the first signal by employing the second signal corrected bythe correction unit and generates an image of the photographic subjectthat has been captured by the first image capturing region.

According to a tenth aspect, a display device comprises: a correctionunit that, for employment in interpolation of a first signal outputtedfrom a first image capturing region of an image capturing element thatcaptures an image of a photographic subject, performs correction upon asecond signal outputted from a second image capturing region of theimage capturing element for which an image capture condition isdifferent from an image capture condition for the first image capturingregion; and a display unit that displays an image of the photographicsubject that has been captured by the first image capturing region,generated by employing a signal generated by interpolating the firstsignal according to the second signal corrected by the correction unit.

According to a 11th aspect, a display device comprises: a correctionunit that, for reducing noise included in a first signal outputted froma first image capturing region of an image capturing element thatcaptures an image of a photographic subject, performs correction upon asecond signal outputted from a second image capturing region of theimage capturing element for which an image capture condition isdifferent from an image capture condition for the first image capturingregion; and a display unit that displays an image of the photographicsubject that has been captured by the first image capturing region,generated by reducing noise included in the first signal by employingthe second signal corrected by the correction unit.

According to a 12th aspect, an image capturing device comprises: animage capturing unit having a first region that captures incident light,and a second region, different from the first region, that capturesincident light; a setting unit that sets an image capture condition forthe first region so as to be different from an image capture conditionfor the second region; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit, based on the imagecapture condition for the second region set by the setting unit; and ageneration unit that generates an image from the first image datacorrected by the correction unit, and second image data generated bycapturing light incident upon the second region.

According to a 13th aspect, an image capturing device comprises: animage capturing unit having a first region that captures incident light,and a second region, different from the first region, that capturesincident light; a setting unit that sets an image capture condition forthe first region so as to be different from an image capture conditionfor the second region; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit based on the imagecapture condition for the second region set by the setting unit, andthat corrects second image data generated by capturing light incidentupon the second region under the image capture condition set by thesetting unit based on the image capture condition for the first regionset by the setting unit; and a generation unit that generates an imagefrom the first image data corrected by the correction unit, and thesecond image data corrected by the correction unit.

According to a 14th aspect, an image processing device comprises: asetting unit that sets an image capture condition for a first region ofan image capturing unit so as to be different from an image capturecondition for a second region of the image capturing unit; a correctionunit that corrects first image data generated by capturing lightincident upon the first region under the image capture condition set bythe setting unit, based on the image capture condition for the secondregion set by the setting unit; and a generation unit that generates animage from the first image data corrected by the correction unit, andsecond image data generated by capturing light incident upon the secondregion.

According to a 15th aspect, an image processing device comprises: asetting unit that sets an image capture condition for a first region ofan image capturing unit so as to be different from an image capturecondition for a second region of the image capturing unit; a correctionunit that corrects first image data generated by capturing lightincident upon the first region under the image capture condition set bythe setting unit based on the image capture condition for the secondregion set by the setting unit, and corrects second image data generatedby capturing light incident upon the second region under the imagecapture condition set by the setting unit based on the image capturecondition for the first region set by the setting unit; and a generationunit that generates an image from the first image data corrected by thecorrection unit, and the second image data generated by capturing lightincident upon the second region.

According to a 16th aspect, a display device comprises: a setting unitthat sets an image capture condition for a first region of an imagecapturing unit so as to be different from an image capture condition fora second region of the image capturing unit; a correction unit thatcorrects first image data generated by capturing light incident upon thefirst region under the image capture condition set by the setting unit,based on the image capture condition for the second region set by thesetting unit; and a display unit that displays an image generated fromthe first image data corrected by the correction unit, and second imagedata generated by capturing light incident upon the second region.

According to a 17th aspect, a display device comprises: a setting unitthat sets an image capture condition for a first region of an imagecapturing unit so as to be different from an image capture condition fora second region of the image capturing unit; a correction unit thatcorrects first image data generated by capturing light incident upon thefirst region under the image capture condition set by the setting unitbased on the image capture condition for the second region set by thesetting unit, and corrects second image data generated by capturinglight incident upon the second region under the image capture conditionset by the setting unit based on the image capture condition for thefirst region set by the setting unit; and a display unit that displaysan image generated from the first image data corrected by the correctionunit, and the second image data generated by capturing light incidentupon the second region.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of the structure of acamera according to a first embodiment;

FIG. 2 is a sectional view of a laminated type imaging element;

FIG. 3 is a figure for explanation of the arrangement of pixels upon animage capturing chip, and of unit regions thereof;

FIG. 4 is a figure for explanation of circuitry for one of the unitregions;

FIG. 5 is a figure schematically showing an image of a photographicsubject that is focused upon the imaging element of the camera;

FIG. 6 is a figure showing an example of an image capture conditionssetting screen;

FIG. 7A is a figure showing an example of the neighborhood of a boundaryof a first region in a live view image, FIG. 7B is a figure showing theneighborhood of the boundary in an enlarged view, and FIG. 7C is anenlarged view of a pixel for attention and of reference pixels;

FIG. 8A is a figure showing an example of an arrangement ofphotoelectrically converted signals outputted from the pixels, FIG. 8Bis a figure for explanation of interpolation of the G color component,and FIG. 8C is a figure showing an example of the image data for the Gcolor component after interpolation;

FIG. 9A is a figure in which the image data of the R color component hasbeen extracted from FIG. 8A, FIG. 9B is a figure for explanation ofinterpolation of the color difference component Cr, and FIG. 9C is afigure for explanation of interpolation of the image data of the colordifference component Cr;

FIG. 10A is a figure in which the image data of the B color componenthas been extracted from FIG. 8A, FIG. 10B is a figure for explanation ofinterpolation of the color difference component Cb, and FIG. 10C is afigure for explanation of interpolation of the image data of the colordifference component Cb;

FIG. 11 is a figure showing an example of the positioning of pixels forfocus detection upon the image capturing surface;

FIG. 12 is a figure showing a partial region of a focus detection pixelline in an enlarged view;

FIG. 13 is a figure showing a focus detection position in an enlargedview;

FIG. 14A is a figure showing an example of a template image representingan object that is to be detected, and FIG. 14B is a figure showing anexample of a live view image and a search range;

FIG. 15 is a flow chart for explanation of the flow of image captureprocessing for setting image capture conditions for each regionindividually;

FIGS. 16A through 16C are figures showing various examples ofarrangement of a first region and a second region upon the imagecapturing surface of the imaging element;

FIG. 17 is a block diagram showing an example of the structure of animage capturing system according to a Variant Embodiment #11;

FIG. 18 is a figure for explanation of provision of a program to amobile device;

FIG. 19 is a block diagram showing an example of the structure of acamera according to a second embodiment;

FIG. 20 is a figure schematically showing a correspondence relationshipin this second embodiment between a plurality of blocks and a pluralityof correction units;

FIG. 21 is a sectional view of a laminated type imaging element;

FIG. 22 is a figure relating to image processing, in which theprocessing of first image data and the processing of second image dataare schematically illustrated;

FIG. 23 is a figure relating to focus detection processing, in which theprocessing of the first image data and the processing of the secondimage data are schematically illustrated;

FIG. 24 is a figure relating to photographic subject detectionprocessing, in which the processing of the first image data and theprocessing of the second image data are schematically shown;

FIG. 25 is a figure relating to the setting of image capture conditionssuch as exposure calculation processing and so on, in which theprocessing of the first image data and the processing of the secondimage data are schematically illustrated;

FIG. 26 is a figure in which processing of first image data andprocessing of second image data according to a Variant Embodiment #13are schematically shown;

FIG. 27 is another figure in which processing of first image data andprocessing of second image data according to Variant Embodiment #13 areschematically shown;

FIG. 28 is yet another figure in which processing of first image dataand processing of second image data according to Variant Embodiment #13are schematically shown; and

FIG. 29 is still another figure in which processing of first image dataand processing of second image data according to Variant Embodiment #13are schematically shown.

DESCRIPTION OF EMBODIMENTS First Embodiment

As one example of an electronic device that is equipped with an imageprocessing device according to a first embodiment of the presentinvention, a digital camera will now be explained. This camera 1 (referto FIG. 1 ) is built so as to be capable of performing image captureunder different conditions for each of a plurality of regions of theimage capturing surface of its imaging element (or image capturingelement) 32 a. An image processing unit 33 performs respectivelyappropriate processing for each of the various regions whose imagecapture conditions are different. The details of a camera 1 of this typewill now be explained with reference to the drawings.

Explanation of Camera

FIG. 1 is a block diagram showing an example of the structure of thecamera 1 according to this first embodiment. In FIG. 1 , the camera 1comprises an image capturing optical system 31, an image capturing unit(or an imaging unit) 32, the aforementioned image processing unit 33, acontrol unit 34, a display unit 35, actuation members 36, and arecording unit 37.

The image capturing optical system 31 conducts light flux from thephotographic field to the imaging unit 32. This imaging unit 32 includesthe imaging element 32 a and a drive unit 32 b, and photoelectricallyconverts an image of the photographic subject that has been formed bythe image capturing optical system 31. The imaging unit 32 is capable ofcapturing the entire area of the image capturing surface of the imagingelement 32 a under the same conditions, and is also capable of capturingeach of various regions of the image capturing surface of the imagingelement 32 a under conditions that are mutually different. The detailsof the imaging unit 32 will be described hereinafter. And the drive unit32 b generates a drive signal that is required for causing the imagingelement 32 a to perform charge accumulation control. Image capturingcommands to the imaging unit 32 for charge accumulation time and so onare transmitted from the control unit 34 to the drive unit 32 b.

The image processing unit 33 comprises an input unit 33 a, a correctionunit 33 b, and a generation unit 33 c. Image data acquired by theimaging unit 32 is inputted to the input unit 33 a. The correction unit33 b performs pre-processing for correction of the image data that hasbeen inputted as described above. The details of this pre-processingwill be described hereinafter. And the generation unit 33 c generates animage on the basis of the above described data that has been inputtedand on the basis of the image data after pre-processing. Moreover, thegeneration unit 33 c performs image processing upon the image data. Thisimage processing may, for example, include color interpolationprocessing, pixel defect correction processing, contour emphasisprocessing, noise reduction processing, white balance adjustmentprocessing, gamma correction processing, display brightness adjustmentprocessing, saturation adjustment processing, and/or the like.Furthermore, the generation unit 33 c generates an image for display bythe display unit 35.

The control unit 34 is, for example, constituted of a CPU, and controlsthe overall operation of the camera 1. For example, the control unit 34may perform predetermined exposure calculation on the basis of thephotoelectrically converted signals acquired by the image capturing unit32, thus determining exposure conditions required for the imagingelement 32 a to perform appropriate exposure, such as chargeaccumulation time (i.e. exposure time), an aperture value for the imagecapturing optical system 31, ISO sensitivities, and so on, and may sendcorresponding commands to the drive unit 32 b. Furthermore, according tothe scene imaging mode set on the camera 1 and/or the types ofphotographic subject elements that have been detected, the control unit34 may determine image processing conditions for adjustment ofsaturation, contrast, sharpness, and so on, and may send correspondingcommands to the image processing unit 33. This detection of photographicsubject elements will be described hereinafter.

The control unit 34 comprises an object detection unit 34 a, a settingunit 34 b, an image capturing control unit 34 c, and an AF calculationunit 34 d. These units may be implemented in software by the controlunit 34 executing a program stored in a non volatile memory not shown inthe figures, or may be implemented with ASICs or the like.

From the image acquired by the image capturing unit 32, by performingper se known object recognition processing, the object detection unit 34a detects photographic subject elements such as people (i.e. humanfaces), animals such as dogs or cats or the like (i.e. animal faces),plants, vehicles such as bicycles, automobiles, trains or the like,stationary objects such as buildings, scenery elements such asmountains, clouds or the like, and/or objects whose specifications havebeen determined in advance. And the setting unit 34 b divides theimaging screen at the image capturing unit 32 into a plurality ofregions that include these photographic subject elements that have beendetected as described above.

Furthermore, the setting unit 34 b sets image capture conditions foreach of this plurality of regions. Such image capture conditions mayinclude the exposure conditions described above (charge accumulationtime, ISO sensitivity, frame rate, and so on) and the image processingconditions described above (for example, a parameter for white balanceadjustment, a gamma correction curve, a parameter for display brightnessadjustment, a saturation adjustment parameter, and so on). It should beunderstood that it would be possible to set the same image captureconditions for all of the plurality of regions; or, alternatively, itwould be also possible to set different image capture conditions foreach different region of the plurality of regions.

The image capturing control unit 34 c controls the image capturing unit32 (i.e. the imaging element 32 a) and the image processing unit 33 byapplying the image capture conditions that have been set for each of theregions by the setting unit 34 b. Due to this, it is possible to causethe image capturing unit 32 to perform image capture under exposureconditions that are different for each of the plurality of regions, andit is also possible to cause the image processing unit 33 to performimage processing under image processing conditions that are differentfor each of the plurality of regions. Any number of pixels may beincluded in each region; for example a region may include 1000 pixels,or a region may include only 1 pixel. Moreover, the numbers of pixels indifferent regions may be different.

The AF calculation unit 34 d controls the automatic focus adjustmentoperation (auto-focus: AF) to a predetermined position upon the imagecapturing screen (hereinafter termed the focus detection position), soas to adjust the focus to the corresponding photographic subject. And,on the basis of the result of this calculation, the AF calculation unit34 d sends a drive signal to the drive unit 32 b for shifting a focusinglens of the image capturing optical system 31 to an appropriate focusingposition. The processing that the AF calculation unit 34 d performs forautomatic focus adjustment is termed “focus detection processing”. Thedetails of this focus detection processing will be describedhereinafter.

The display unit 35 reproduces and displays images that have beengenerated by the image processing unit 33, images that have been imageprocessed, images that have been read out by the recording unit 37, andso on. And the display unit 35 also displays an actuation menu screen, asetting screen for setting image capture conditions, and so on.

The actuation members 36 include actuation members of various types,such as a release button and a menu button and so on. And, correspondingto actuations of various types, the actuation members 36 send actuationsignals to the control unit 34. The actuation members 36 also include atouch actuation member that is provided to a display surface of thedisplay unit 35.

According to a command from the control unit 34, the recording unit 37records image data and so on upon a recording medium consisting of amemory card or the like, not shown in the figures. Moreover, accordingto a command from the control unit 34, the recording unit 37 also readsout image data recorded upon the recording medium.

Explanation of the Laminated Type Imaging Element

As one example of the imaging element 32 a described above, a laminatedor stacked imaging element 100 will now be explained. FIG. 2 is asectional view of this imaging element 100. The imaging element 100comprises an image capturing chip 111, a signal processing chip 112, anda memory chip 113. The image capturing chip 111 is laminated to thesignal processing chip 112. And the signal processing chip 112 islaminated to the memory chip 113. The image capturing chip 111 and thesignal processing chip 112, and similarly the signal processing chip 112and the memory chip 113, are electrically connected together byconnecting portions 109. These connecting portions 109 may, for example,be bumps or electrodes. The image capturing chip 111 captures an opticalimage from the photographic subject, and generates image data. And theimage capturing chip 111 outputs this image data to the signalprocessing chip 112 from the image capturing chip 111. The signalprocessing chip 112 performs signal processing on the image dataoutputted from the image capturing chip 111. Moreover, the memory chip113 comprises a plurality of memories, and stores image data. It shouldbe understood that it would also be acceptable for the imaging element100 to comprise only an image capturing chip and a signal processingchip. If the imaging element 100 thus comprises only an image capturingchip and a signal processing chip, then a storage unit for storage ofthe image data may be provided at the signal processing chip, or may beprovided separately from the imaging element 100.

As shown in FIG. 2 , the incident light is mainly incident in the +Zaxis direction, as shown by the white arrow sign. Moreover, as shown bythe coordinate axes in the figure, the direction orthogonal to the Zaxis and leftward on the drawing paper is taken as being the +X axisdirection, and the direction orthogonal to both the Z axis and the Xaxis and toward the viewer from the drawing paper is taken as being the+Y axis direction. Coordinate axes are shown in some of the subsequentfigures, so that the orientation of those figures with reference to thecoordinate axes of FIG. 2 can be understood.

The image capturing chip 11 may, for example, be a CMOS image sensor. Inconcrete terms, the image capturing chip 111 may be a CMOS image sensorof the backside illumination type. The image capturing chip 111comprises a micro-lens layer 101, a color filter layer 102, apassivation layer 103, a semiconductor layer 106, and a wiring layer108. And, in the image capturing chip 111, the micro-lens layer 101, thecolor filter layer 102, the passivation layer 103, the semiconductorlayer 106, and the wiring layer 108 are arranged in that order along the+Z axis direction.

The micro-lens layer 101 includes a plurality of micro-lenses L. Themicro-lenses L condense the incident light onto photoelectric conversionunits 104 that will be described hereinafter. The color filter layer 102includes a plurality of color filters F. That is, the color filter layer102 includes color filters F of a plurality of types having differentspectral characteristics. In concrete terms, the color filter layer 102includes first filters (R) that have the spectral characteristic ofprincipally passing light having a red color component, second filters(Gb and Gr) that have the spectral characteristic of principally passinglight having a green color component, and third filters (B) that havethe spectral characteristic of principally passing light having a bluecolor component. In the color filter layer 102, for example, the firstfilters, the second filters, and the third filters may be arranged as aBayer array. And the passivation layer 103 is formed as a nitride filmor an oxide film, and protects the semiconductor layer 106.

The semiconductor layer 106 includes photoelectric conversion units 104and readout circuits 105. In detail, the semiconductor layer 106comprises a plurality of photoelectric conversion units 104 between itsfirst surface 106 a, which is its surface upon which light is incident,and its second surface 106 b, which is its surface on the side oppositeto the first surface 106. In the semiconductor layer 106, the pluralityof photoelectric conversion units 104 are arranged along the X axisdirection and along the Y axis direction. The photoelectric conversionunits 104 are endowed with a photoelectric conversion function ofconverting light into electrical charge. Moreover, the photoelectricconversion units 104 accumulate the charges of these photoelectricallyconverted signals. The photoelectric conversion units 104 may, forexample, be photo-diodes. The semiconductor layer 106 contains thereadout circuits 105 in positions closer to its second surface 106 bthan the photoelectric conversion units 104. The readout circuits 105are arranged in the semiconductor layer 106 along the X axis directionand the Y axis direction in the semiconductor layer 106. Each of thereadout circuits 105 is built from a plurality of transistors, and readsout and outputs to the wiring layer 108 the image data generated by thecharges that have been photoelectrically converted by the correspondingphotoelectric conversion unit 104.

The wiring layer 108 comprises a plurality of metallic layers. Themetallic layers may, for example, be Al wiring or Cu wiring or the like.Image data that has been read out by the readout circuits 105 isoutputted via the wiring layer 108. This image data is outputted fromthe wiring layer 108 to the signal processing chip 112 via theconnecting portions 109.

It should be understood that one of the connecting portions 109 may beprovided for each of the photoelectric conversion units 104. Moreover,it would also be acceptable for each one of the connecting portions 109to be provided to a group of the photoelectric conversion units 104. Ifeach of the connection portions 109 is provided to a group of thephotoelectric conversion units 104, then the pitch of the connectingportions 109 may be greater than the pitch of the photoelectricconversion units 104. Furthermore, the connecting portions 109 may beprovided in a region that is peripheral to the region where thephotoelectric conversion units 104 are disposed.

The signal processing chip 112 comprises a plurality of signalprocessing circuits. These signal processing circuits perform signalprocessing on the captured image data outputted from the image capturingchip 111. The signal processing circuits may, for example, be amplifiercircuits that amplify the signal values of the captured image data,correlated double sampling circuits that perform noise reductionprocessing on the image data, analog/digital (A/D) conversion circuitsthat convert analog signals to digital signals, and so on. One of thesignal processing circuits may be provided for each of the photoelectricconversion units 104.

Furthermore, each of the signal processing circuits may be provided to agroup of the photoelectric conversion units 104. The signal processingchip 112 has a plurality of through electrodes or vias 110. These vias110 may, for example, be through-silicon vias. The vias 110 connectcircuits that are provided on the signal processing chip 112 to oneanother. The vias 110 may also be provided to the peripheral regions ofthe image capturing chip 111 and to the memory chip 113. It should beunderstood that it would also be acceptable to provide some of theelements included in the signal processing circuit on the imagecapturing chip 111. For example, in the case of the analog/digitalcircuit, a comparator that performs comparison of the input voltage to areference voltage may be provided at the image capturing chip 111, andcircuitry such as a counter circuit and/or a latch circuit and so on maybe provided at the signal processing chip 112.

The memory chip 113 comprises a plurality of storage units. Thesestorage units store image data that has been subjected to signalprocessing by the signal processing chip 112. The storage units may, forexample, be volatile memories such as DRAMs or the like. One of thestorage units may be provided for each of the photoelectric conversionunits 104. Alternatively, each one of the storage units may be providedto a group of the photoelectric conversion units 104. The image datastored in the storage units is outputted to the image processing unit ata subsequent stage.

FIG. 3 is a figure for explanation of the arrangement of pixels on theimage capturing chip 111, and for explanation of unit regions 131thereof. In particular, this figure shows a situation in which the imagecapturing chip 111 is being viewed from its rear surface (i.e. from itsimage capturing surface). In the pixel region, for example, at least 20million pixels may be arranged in the form of a matrix. In the FIG. 3example, four adjacent pixels constitute a single unit region 131 thatis 2 pixels×2 pixels. The lattice grid in the figure illustrates theconcept that adjacent pixels are grouped together to form the unitregions 131. The number of pixels that constitute one unit region 131 isnot limited to being four as above; it would be acceptable for thisnumber to be around a thousand, for example for the unit region to be 32pixels×32 pixels; and the number could be greater than that or less thanthat; the unit region could even be a single pixel.

As shown in the partial enlarged view of the pixel region, the unitregion 131 in FIG. 3 is formed as a so-called Bayer array that consistsof two green color pixels Gb and Gr, a blue color pixel B, and a redcolor pixel R. The green color pixels Gb and Gr are pixels that havegreen color filters as their color filters F, and that receive light ofgreen color wavelength in the incident light. In a similar manner, theblue color pixels B are pixels that have blue color filters as theircolor filters F, and that receive light of blue color wavelength in theincident light, and the red color pixels R are pixels that have redcolor filters as their color filters F, and that receive light of redcolor wavelength in the incident light.

In this embodiment, a plurality of blocks are defined so that at leastone of the unit regions 131 is included in each block. In other words,the minimum unit in each block is a single unit region 131. As describedabove, among the values that can be taken as the number of pixelsforming a single unit region 131, the smallest number of pixels is asingle pixel. Accordingly, if one block is defined in terms of pixelunits, among the number of pixels that can define one block, the minimumnumber of pixels is a single pixel. The pixels that are included in oneblock can be controlled with control parameters that are different fromanother block. That is, in each block, all of the unit regions 131within that block, in other words all of the pixels within that block,are controlled with the same image capture conditions. In other words,photoelectrically converted signals for which the image captureconditions are different can be acquired for the pixel group included inone block and for the pixel group included in a different block.Examples of control parameters are frame rate, gain, decimation ratio,number of rows or number of columns for adding the photoelectricallyconverted signals, charge accumulation time or accumulation number,number of bits for digitization (i.e. word length), and so on. Theimaging element 100 not only can freely perform decimation in the rowdirection (i.e. in the X axis direction of the image capturing chip111), but can also freely perform decimation in the column direction(i.e. in the Y axis direction of the image capturing chip 111).Furthermore, the control parameter may include parameters for the imageprocessing.

FIG. 4 is a figure for explanation of the circuitry for a single unitregion 131. In the FIG. 4 example, a single unit region 131 consists offour adjacent pixels, that is 2 pixels×2 pixels. It should be understoodthat the number of pixels included in one unit region 131 is not limitedto being four as above; it would be acceptable for this number to be athousand or more, and at a minimum it could even be a single pixel. Thetwo dimensional positions in the unit region 131 are designated in FIG.4 by the reference symbols A through D.

Reset transistors (RST) of the pixels included in the unit region 131are adapted to be capable of being turned on and off individually foreach pixel. In FIG. 4 , reset wiring 300 is provided for turning thereset transistor of the pixel A on and off, and reset wiring 310 forturning the reset transistor of the pixel B on and off is providedseparately from the above described reset wiring 300. Similarly, resetwiring 320 for turning the reset transistor of the pixel C on and off isprovided separately from the reset wiring 300 and the reset wiring 310.And, similarly, dedicated reset wiring 330 is also provided for turningthe reset transistor of the other pixel D on and off.

Transfer transistors (TX) of the pixels included in the unit region 131are also adapted to be capable of being turned on and off individuallyfor each pixel. In FIG. 4 , transfer wiring 302 for turning the transfertransistor of the pixel A on and off, transfer wiring 312 for turningthe transfer transistor of the pixel B on and off, and transfer wiring322 for turning the transfer transistor of the pixel C on and off areprovided separately. And dedicated transfer wiring 332 is also providedfor turning the transfer transistor of the other pixel D on and off.

Furthermore, selection transistors (SEL) of the pixels included in theunit region 131 are also adapted to be capable of being turned on andoff individually for each pixel. In FIG. 4 , selection wiring 306 forturning the selection transistor of the pixel A on and off, selectionwiring 316 for turning the selection transistor of the pixel B on andoff, and selection wiring 326 for turning the selection transistor ofthe pixel C on and off are provided separately. And dedicated selectionwiring 336 is also provided for turning the selection transistor of theother pixel D on and off.

It should be understood that power supply wiring 304 is connected incommon to all the pixels A through D included in the unit region 131. Ina similar manner, output wiring 308 is connected in common to all thepixels A through D included in the unit region 131. Here, while thepower supply wiring 304 is connected in common between a plurality ofthe unit regions, the output wiring 308 is provided separately for eachof the unit regions 131. A load current source 309 supplies current tothe output wiring 308. This load current source 309 may be provided atthe image capturing chip 111, or may be provided at the signalprocessing chip 112.

By turning the reset transistors and the transfer transistors of theunit region 131 on and off individually, charge accumulation for thepixels A through D included in the unit region 131 can be controlled,with this control including their charge accumulation start timings,their charge accumulation end timings, and their transfer timings.Moreover, the photoelectrically converted signals from the pixels Athrough D can be outputted via the common output wiring 308 by turningthe selection transistors of the unit region 131 individually on andoff.

Here, a so called rolling shutter method is per se known for controllingcharge accumulation by the pixels A through D included in the unitregions 131 in a regular sequence according to rows and columns. Whencolumns are designated after pixels in each row are selected accordingto such a rolling shutter method, then, in the example of FIG. 4 , thephotoelectric conversion signals are outputted in the sequence “ABCD”.

By constructing the circuitry based upon each of the unit regions 131 inthis manner, the charge accumulation time for each unit region 131 canbe individually controlled. To put it in another manner,photoelectrically converted signals at frame rates that are individuallydifferent for the respective unit regions 131 can be outputted.Furthermore, by performing charge accumulation (i.e. image capture) forthe unit regions 131 that are included in a certain predetermined blockon the image capturing chip 111, while idling the unit regions that areincluded in another block thereof, it is possible to perform imagecapture only for the certain predetermined block on the image capturingchip 111, and to output their photoelectrically converted signals. Yetfurther, it is also possible to output photoelectrically convertedsignals while changing over, between frames, the blocks for which chargeaccumulation (i.e. image capture) is performed (i.e. by changing overthe blocks that are the subjects of charge accumulation control), thusperforming image capture sequentially with different blocks of the imagecapturing chip 111.

As described above, the output wiring 308 is provided to correspond toeach of the unit regions 131 individually. Since, in this imagingelement 100, the image capturing chip 111, the signal processing chip112, and the memory chip 113 are laminated together, accordingly, byemploying electrical connections between the chips by using theconnection portions 109 in the output wiring 308, it is possible toroute the wiring without increasing the sizes of the chips in thesurface direction.

Block Control of the Imaging Element

In this embodiment, the image capture conditions are settableindividually for each of the plurality of blocks on the imaging element32 a. The control unit 34 (i.e. the image capturing control unit 34 c)establishes correspondence between the plurality of regions describedabove and the blocks described above, and performs image capture foreach region under the image capture conditions that have been set forthat region.

FIG. 5 is a figure schematically showing an image of a photographicsubject that has been formed on the image capturing element 32 a of thecamera 1. Before the image capture command is executed, the camera 1acquires a live view image by photoelectrically converting the image ofthe photographic subject. The term “live view image” refers to an imagefor monitoring that is repeatedly captured at a predetermined frame rate(for example at 60 fps).

Before the division into regions by the setting unit 34 b, the controlunit 34 sets the same image capture conditions for the entire area ofthe image capturing chip 111 (in other words, for the entire imagingscreen). Here, the term “the same image capture conditions” refers toimage capture conditions that are set in common for the entire imagingscreen. The conditions in which, for example, the apex values vary byless than approximately 0.3 levels may be considered to be the same.These image capture conditions that are set to be the same over theentire area of the image capturing chip 111 are determined on the basisof exposure conditions corresponding to the photometric value of theluminance of the photographic subject, or on the basis of exposureconditions that have been set manually by the user.

In FIG. 5 , an image that includes a person 61 a, an automobile 62 a, abag 63 a, mountains 64 a, and clouds 65 a and 66 a is shown as formed onthe image capturing surface of the image capturing chip 111. The person61 a is holding the bag 63 a with both hands. And the automobile 62 a isstopped behind the person 61 a and to her right.

Division into Regions

On the basis of the live view image, the control unit 34 divides thescreen of the live view image into a plurality of regions in thefollowing manner. First, elements of the photographic subject aredetected from the live view image by the object detection unit 34 a.This detection of the photographic subject elements employs a per seknown photographic subject recognition technique. In the example of FIG.5 , the object detection unit 34 a detects the person 61 a, theautomobile 62 a, the bag 63 a, the mountains 64 a, and the clouds 65 aand 66 a as photographic subject elements.

Next, the live view image screen is subdivided by the setting unit 34 binto regions that include the photographic subject elements describedabove. The explanation of this embodiment will suppose that a regionincluding a person 61 a is defined as being a region 61, a regionincluding an automobile 62 a is defined as being a region 62, a regionincluding a bag 63 a is defined as being a region 63, a region includingmountains 64 a is defined as being a region 64, a region including acloud 65 a is defined as being a region 65, and a region including acloud 66 a is defined as being a region 66.

Setting of the Image Capture Conditions for Each Block

When the screen has been divided into a plurality of regions by thesetting unit 34 b, the control unit 34 causes the display unit 35 todisplay a setting screen like the example shown in FIG. 6 . FIG. 6 showsdisplay of a live view image 60 a, and moreover an image captureconditions setting screen 70 is displayed to the right of the live viewimage 60 a.

On the setting screen 70, as an example of image capture conditionssetting items, frame rate, shutter speed (TV), and gain (ISO) are shownlisted in that order from the top. The frame rate is the number offrames of the live view image acquired in one second, or the number offrames of a moving video image recorded by the camera 1 in one second.The gain is ISO sensitivity. In addition to the image capture conditionsetting items shown in the FIG. 6 example, other suitable setting itemsmay be included as appropriate. If all the setting items cannot befitted into the setting screen 70, then it will be acceptable to arrangefor the other setting items to be displayed by scrolling the settingitems up and down.

In this embodiment, among the regions that have been divided by thesetting unit 34 b, the control unit 34 takes a region that has beenselected by the user as the subject of setting (or changing) its imagecapture conditions. For example, in the case of this camera 1 that iscapable of being operated by touch actuation, the user may performtapping actuation on the display surface of the display unit 35, onwhich the live view image 60 a is being displayed, at the position wherethe image of the main photographic subject, for which he wishes to set(or change) the image capture conditions, is being displayed. If, forexample, tapping actuation has been performed by the user on theposition where the image of the person 61 a is being displayed, then thecontrol unit 34 takes the region 61 in the live view image 60 a thatincludes the person 61 a as being the subject region for setting (or forchanging) the image capture conditions, and also accentuates the displayof the contour of the region 61.

In FIG. 6 , the region 61 whose contour is displayed as accentuated (bybeing displayed with a thick outline, by being displayed as brighter, bybeing displayed with its color altered, by being displayed as surroundedby a broken line, by being displayed as blinking, or the like) indicatesthe region that is the subject of setting (or changing) its imagecapture conditions. In the example of FIG. 6 , it will be supposed thatthe live view image 60 a is being displayed with the contour of theregion 61 accentuated. In this case, the region 61 is the subject ofsetting (or of changing) its image capture conditions. For example when,in the case of this camera 1 that is capable of being operated by touchactuation, tapping actuation is performed by the user on the display 71of shutter speed (TV), then the control unit 34 causes the currently setvalue of shutter speed for the region that is being displayed asaccentuated (i.e. for the region 61) to be displayed within the screen(as shown by a reference symbol 68).

In the following description, the explanation will suppose that thecamera 1 is operated by touch actuation, but it would also be acceptableto arrange for setting (or changing) of the image capture conditions tobe performed by actuation of one or more buttons included in theactuation members 36 or the like.

When tapping actuation is performed by the user on an upper icon 71 a oron a lower icon 71 b for shutter speed (TV), the setting unit 34 bincreases or decreases the displayed value 68 of shutter speed from itscurrently set value according to this tapping actuation, and also sendsa command to the image capturing unit 32 (refer to FIG. 1 ) so as tocause the image capturing unit 32 to change the image capture conditionsfor the unit regions 131 (refer to FIG. 3 ) of the imaging element 32 athat correspond to the region that is currently being displayed asaccentuated (i.e. for the unit regions of the region 61), according tothe above described tapping actuation. A confirm icon 72 is an actuationicon for confirming the image capture conditions that have been set.And, in a similar manner to the case of setting (or changing) theshutter speed (TV), the setting unit 34 b also performs setting (orchanging) of the frame rate and of the gain (ISO).

Although in the above explanation a case has been described in which thesetting unit 34 b performs setting of the image capture conditions onthe basis of actuation by the user, it should be understood that thisfeature is not intended to be limitative. It will also be acceptable toarrange for the setting unit 34 b not to set the image captureconditions on the basis of actuation by the user, but according todecision by the control unit 34. For example, if blown-out highlights orblocked-up shadows take place in a region of the image that includes aphotographic subject element for which the luminance is maximum orminimum, then it will be acceptable to arrange for the setting unit 34 bto set the image capture conditions so as to eliminate such blown-outhighlights or blocked-up shadows, according to decision by the controlunit 34.

For the regions that are not displayed as accentuated (i.e. the regionsother than the region 61), the set image capture conditions aremaintained without alteration.

Instead of displaying as accentuated the contour of the region that isthe subject of setting (or of changing) its image capture conditions, itwould also be acceptable to arrange for the control unit 34 to displaythat entire subject region with its brightness increased, or to displaythat entire subject region with its contrast increased, or to displaythat entire subject region as blinking. Moreover, it would also beacceptable to surround the subject region with a frame. Such a framethat is displayed as surrounding the subject region may be a doubleframe or a single frame, and the display style for such a surroundingframe, such as its line type, its color, its brightness, or the like,may be varied as appropriate. Furthermore, it would also be acceptableto arrange for the control unit 34 to provide, in the neighborhood ofthe subject region, a display of an arrow sign or the like thatindicates that this is the region that is the subject of image captureconditions setting. It would also be acceptable to arrange for thecontrol unit 34 to display the regions other than the subject regionthat is the subject of image capture conditions setting (or changing) asdarkened, or to display such regions other than the subject region withtheir contrast reduced.

After the image capture conditions for each region have been set asexplained above, when a release button not shown in the figures that isincluded in the actuation members 36 is actuated, or when a display thatcommands the start of image capture (such as a shutter release icon) isactuated, image capture is performed under the image capture conditionsthat were respectively set for the above described subdivided regions,by the imaging unit 32 controlled by the control unit 34. And the imageprocessing unit 33 performs image processing upon the image data thathas been acquired by the imaging unit 32. As described above, this imageprocessing may be performed under image processing conditions that aredifferent for each of the regions.

In response to a command from the control unit 34 after the abovedescribed image processing by the image processing unit 33, therecording unit 37 records the image data after image processing upon arecording medium constituted of a memory card or the like, not shown inthe figures. After this, the image capture processing terminates.

Correction Processing

As described above, in this first embodiment, it is arranged that, afterthe regions on the image capture screen have been subdivided by thesetting unit 34 b, the image capture conditions can be set (or changed)for a region selected by the user, or for a region determined by thecontrol unit 34. If different image capture conditions are set for thedifferent subdivided regions, the control unit 34 will perform thefollowing correction processing as necessary.

1. When Performing Image Processing

In the case in which the image processing to be performed on the imagedata acquired by applying the image capture conditions that aredifferent for each of the subdivided regions is predetermined imageprocessing, then, as pre-processing for the image processing, the imageprocessing unit 33 (i.e. the correction unit 33 b) performs correctionprocessing on the image data that is positioned in the vicinity ofboundary regions. The predetermined image processing is processing forcalculating image data for a position for attention in the image that isthe subject for processing with reference to image data of a pluralityof reference positions surrounding the position for attention, and forexample may include pixel defect correction processing, colorinterpolation processing, contour emphasis processing, noise reductionprocessing, and so on.

Correction processing is performed in order to alleviate discontinuitygenerated in the image after the image processing, originating in thefact that the image capture conditions are different between thedifferent subdivided regions. In general, if the position for attentionis located in the vicinity of a boundary between divided regions, bothof image data to which the same image capture conditions as those forthe image data at the position of attention have been applied, and imagedata to which different image capture conditions from those for theimage data at the position of attention have been applied, may bepresent at a plurality of reference positions around the position forattention. In this embodiment, rather than calculating the image datafor the position for attention by referring to the image data for thereference positions to which different image capture conditions havebeen applied just as it is without alteration, correction processing isperformed as will now be described, based upon the consideration that itis more desirable to calculate the image data at the position forattention by referring to image data at the reference positions on whichcorrection processing has been performed in order to suppress variancebetween sets of image data caused by the different image captureconditions.

FIG. 7A is a figure showing an example of a region 80 in the live viewimage 60 a in the vicinity of a boundary between the region 61 and theregion 64. In this example, it will be supposed that first image captureconditions are set to the region 61 that includes at least one person,and that second image capture conditions are set to the region 64 thatincludes mountains. And FIG. 7B is a figure showing the region 80 ofFIG. 7A in the vicinity of the boundary in an enlarged view. Image datafrom pixels in the imaging element 32 a corresponding to the region 61for which the first image capture conditions are set is shown with awhite background, while image data from pixels in the imaging element 32a corresponding to the region 64 for which the second image captureconditions are set is shown by shading. In FIG. 7B, the image data fromthe pixel for attention P in the region 61 is located in the vicinity ofthe boundary 81 between the region 61 and the region 64, in other wordsat their boundary portion. Pixels (in this example, eight pixels) thatare included in a predetermined range 90 around the pixel for attentionP (in this example, this range is 3×3 pixels in size) centered upon thatpixel for attention P are taken as being reference pixels. Moreover,FIG. 7C is an enlarged view of the pixel for attention P and itsreference pixels. The position of the pixel for attention P is theposition for attention, and the positions of the reference pixelssurrounding the pixel for attention P are the reference positions.

Normally, the image processing unit 33 (i.e. the generation unit 33 c)performs image processing by referring directly to the image data of thereference pixels just as it is, without performing any correctionprocessing. However, if the image capture conditions that were appliedto the pixel for attention P (which are referred to as the first imagecapture conditions) and the image capture conditions that were appliedto reference pixels surrounding the pixel for attention P (which arereferred to as the second image capture conditions) are different, thenthe correction unit 33 b performs correction processing as shown by thefollowing Examples #1 through #3 on the image data for which the secondimage capture conditions were applied among the image data for thereference pixels. And the generation unit 33 c performs image processingfor calculating image data for the pixel for attention P by referring tothe reference pixels after correction processing. In FIG. 7C, the imagedata outputted from the pixels shown with white background is image datathat has been captured under the first image capture conditions, whilethe image data outputted from the pixels shown by shading is image datathat has been captured under the second image capture conditions.

Example #1

If, for example, the only difference between the first image captureconditions and the second image capture conditions is the ISOsensitivity, with the ISO sensitivity of the first image captureconditions being 100 while the ISO sensitivity of the second imagecapture conditions is 800, then the image processing unit 33 (i.e. thecorrection unit 33 b) performs multiplication by 100/800 as thecorrection processing for the image data that has been captured underthe second image capture conditions in the image data for the referencepixels. By doing this, variance between the sets of the image datacaused by the different image capture conditions is reduced.

Although the variance in the image data is small if the amount of lightincident upon the pixel for attention P and the amounts of lightincident upon the reference pixels are the same, it should be understoodthat, if originally the amount of incident light on the pixel forattention P and the amounts of incident light on the reference pixelsare different, the variance in the image data may not become small. Thesame is the case in the examples described hereinafter.

Example #2

If, for example, the only difference between the first image captureconditions and the second image capture conditions is the shutter speed,with the shutter speed of the first image capture conditions being1/1000 sec. while the shutter speed of the second image captureconditions is 1/100 sec., the image processing unit 33 (i.e. thecorrection unit 33 b) performs multiplication by 1/1000/ 1/100= 1/10 asthe correction processing for the image data, among the image data forthe reference pixels, that has been captured under the second imagecapture conditions. By doing this, discrepancy in the image data causedby the different image capture conditions is reduced.

Example #3

If, for example, the only difference between the first image captureconditions and the second image capture conditions is the frame rate(with the charge accumulation times being equal), with the frame rate ofthe first image capture conditions being 30 fps while the frame rate ofthe second image capture conditions is 60 fps, the image processing unit33 (i.e. the correction unit 33 b), as correction processing, employsthe image data of those frame images whose acquisition start timings areclosest to those of the frame images that were acquired under the firstimage capture conditions (i.e. 30 fps) in place of the image data thathas been captured under the second image capture conditions (i.e. at 60fps) in the image data for the reference pixels. By doing this, variancein the image data due to difference in the image capture conditions isreduced.

It should be understood that it would also be acceptable to performcorrection processing in which interpolation calculation of the imagedata of frame images whose acquisition start timings are close to thoseof the frame images that were acquired under the first image captureconditions (i.e. at 30 fps) on the basis of a plurality of previous andsubsequent frame images acquired under the second image conditions (i.e.at 60 fps).

On the other hand, if the image capture conditions that were applied tothe pixel for attention P (here supposed to be the first image captureconditions) and the image capture conditions that were applied to all ofthe reference pixels surrounding the pixel for attention P (heresupposed to be the second image capture conditions) are the same, thenthe image processing unit 33 (i.e. the correction unit 33 b) does notperform any correction processing upon the image data for the referencepixels. In other words, the generation unit 33 c performs imageprocessing for calculation of the image data of the pixel for attentionP by referring to the image data for the reference pixels just as it iswithout alteration.

It should be understood that, as mentioned above, even if there arerelatively insubstantial differences in the image capture conditions,they are considered as being the same image capture conditions

Examples of Image Processing

Examples of image processing accompanied by correction processing willnow be described.

(1) Pixel Defect Correction Processing

In this embodiment, pixel defect correction processing is one type ofimage processing that is performed at the time of image capture.Generally, in the image capturing element 32 a that is a solid-stateimage sensor, one or more pixel defects may occur during the process ofmanufacture or after manufacture, so that some data having anomalouslevels may be outputted. Accordingly the image processing unit 33 (i.e.the generation unit 33 c) is adapted, by correcting the image dataoutputted from those pixels for which pixel defects have occurred, toensure that the image data at the pixel positions at which pixel defectshave occurred does not stand out conspicuously.

An example of such pixel defect correction processing will now beexplained. The image processing unit 33 (i.e. the generation unit 33 c)may, for example, take a pixel in the image of one frame that ispositioned at a pixel defect that is recorded in advance in anon-volatile memory (not shown in the figures) as being a pixel forattention P (i.e. as a pixel to be the subject of processing), and maytake pixels (in this example, eight pixels) around the periphery of thepixel for attention P that are included in a predetermined range 90 (forexample 3/3 pixels) centered on the pixel for attention P as beingreference pixels.

The image processing unit 33 (i.e. the generation unit 33 c) calculatesthe maximum values and the minimum values of the image data for thereference pixels, and, if the image data outputted from the pixel forattention P is outside the limits of the maximum value and the minimumvalue, performs so-called “Max, Min” filter processing to replace theimage data outputted from the pixel for attention P with the abovedescribed maximum value or minimum value. This type of processing isperformed for all of the pixel defects for which position information isrecorded in the non-volatile memory (not shown in the figures).

In this embodiment, if some pixels to which the second image captureconditions have been applied that are different from the first imagecapture conditions applied to the pixel for attention P are included inthe reference pixels described above, then the image processing unit 33(i.e. the correction unit 33 b) performs correction processing on theimage data to which the second image capture conditions were applied.Subsequently, the image processing unit 33 (i.e. the generation unit 33c) performs the “Max, Min” filter processing described above.

(2) Color Interpolation Processing

In this embodiment, color interpolation processing is one type of imageprocessing that is performed at the time of image capture. As shown inthe example of FIG. 3 , in the image capturing chip 111 of the imagingelement 100, green color pixels Gb and Gr, blue color pixels B, and redcolor pixels R are arranged in a Bayer array. Since there is a lack ofthe image data of the color component that is different from the colorcomponent of the color filter F disposed at a corresponding pixelposition, accordingly, by referring to the image data for surroundingpixel positions, the image processing unit 33 (i.e. the generation unit33 c) performs color interpolation processing in order to generate imagedata of the color component that is lacking.

An example of such color interpolation processing will now be explained.FIG. 8A is a figure showing an example of the arrangement of image dataoutputted from the imaging element 32 a. Corresponding to each pixelposition, this arrangement has color components of each of the colors R,G, and B, according to the Bayer array rule.

G Color Interpolation

First, the usual G color interpolation will be explained. In performingthe G color interpolation, the image processing unit 33 (i.e. thegeneration unit 33 c) takes the positions of the R color component andthe B color component in order as positions for attention, and generatesimage data for the G color component at these positions for attention byreferring to four items of G color component image data at fourreference positions around these positions for attention. When, forexample, generating image data for the G color component at the positionfor attention shown by the thick frame in FIG. 8B (at the second row andthe second column, counting from the upper left position, andsubsequently the position of attention is specified by counting in asimilar manner from the upper left position), the four G color componentimage data items G1 through G4 positioned in the neighborhood of theposition for attention (at the second row and the second column) arereferred to. For example, the image processing unit 33 (i.e. thegeneration unit 33 c) may take the value (aGl+bG2+cG3+dG4)/4 as beingthe image data for the G color component at the position for attention(i.e. at the second row and the second column). It should be understoodthat “a” through “d” here are weighting coefficients that are providedto correspond to the distances between the reference positions and theposition for attention, and to the structure of the image.

Next, the G color interpolation in this embodiment will be explained. InFIGS. 8A to 8C, it will be supposed that the first image captureconditions are applied to the region that is leftward of and above thethick line, while the second image capture conditions are applied to theregion that is rightward of and below the thick line. And it should beunderstood that, in FIGS. 8A through 8C, the first image captureconditions and the second image capture conditions are different fromone another. Moreover, the image data items G1 through G4 of the G colorcomponent in FIG. 8B are reference positions for the image processingfor the pixel at the position for attention (at the second row and thesecond column). In FIG. 8B, the first image capture conditions areapplied at the position for attention (at the second row and the secondcolumn). And, among the reference positions, the first image captureconditions are applied to the image data items G1 through G3. However,among the reference positions, the second image capture conditions areapplied to the image data item G4. Due to this, the image processingunit 33 (i.e. the generation unit 33 c) performs correction processingon the image data item G4. And subsequently the image processing unit 33(i.e. the generation unit 33 c) calculates the image data of the G colorcomponent at the position for attention (at the second row and thesecond column).

By generating image data of the G color component at each of thepositions of the B color component and at each of the positions of the Rcolor component in FIG. 8A, the image processing unit 33 (i.e. thegeneration unit 33 c) is able to obtain image data of the G colorcomponent at each of the pixel positions, as shown in FIG. 8C.

R Color Interpolation

FIG. 9A is a figure in which the image data of the R color component hasbeen extracted from FIG. 8A. The image processing unit 33 (i.e. thegeneration unit 33 c) calculates the image data of the color differencecomponent Cr shown in FIG. 9B on the basis of the image data for the Gcolor component shown in FIG. 8C and the image data for the R colorcomponent shown in FIG. 9A.

First, the usual interpolation of the color difference component Cr willbe explained. When, for example, generating image data for the colordifference component Cr at the position for attention shown by the thickframe (at the second row and the second column) in FIG. 9B, the imageprocessing unit 33 (i.e. the generation unit 33 c) refers to the fouritems of image data Cr1 through Cr4 of the color difference componentpositioned in the neighborhood of the position for attention (at thesecond row and the second column). For example, the image processingunit 33 (i.e. the generation unit 33 c) may take the value(eCr1+fCr2+gCr3+hCr4)/4 as being the image data for the color differencecomponent Cr at the position for attention (i.e. at the second row andthe second column). It should be understood that “e” through “h” hereare weighting coefficients that are provided to correspond to thedistances between the reference positions and the position forattention, and to the structure of the image.

In a similar manner, when, for example, generating image data for thecolor difference component Cr at the position for attention shown by thethick frame in FIG. 9C (at the second row and the third column), theimage processing unit 33 (i.e. the generation unit 33 c) refers to thefour items of image data Cr2 and Cr4 through Cr6 of the color differencecomponent positioned in the neighborhood of the position for attention(at the second row and the third column). For example, the imageprocessing unit 33 (i.e. the generation unit 33 c) may take the value(qCr2+rCr4+sCr5+tCr6)/4 as being the image data for the color differencecomponent Cr at the position for attention (at the second row and thethird column). It should be understood that “q” through “t” here areweighting coefficients that are provided to correspond to the distancesbetween the reference positions and the position for attention, and tothe structure of the image. In this manner, image data for the colordifference component Cr is generated for all the pixel positions.

Next, the interpolation of the color difference component Cr in thisembodiment will be explained. In FIGS. 9A through 9C, for example, it issupposed that the first image capture conditions are applied to theregion that is leftward of and above the thick line, while the secondimage capture conditions are applied to the region that is rightward ofand below the thick line. It should be understood that, in FIGS. 9Athrough 9C, the first image capture conditions and the second imagecapture conditions are different from one another. In FIG. 9B, theposition shown by the thick frame (at the second row and the secondcolumn) is the position for attention of the color differencecoefficient Cr. Moreover, the image data items Cr1 through Cr4 for thecolor difference component in FIG. 9B are reference positions forperforming image processing upon the pixel at the position for attention(at the second row and the second column). In FIG. 9B, the first imagecapture conditions are applied at the position for attention (at thesecond row and the second column). Among the reference positions, thefirst image capture conditions are applied to the image data items Cr1,Cr3, and Cr4. Moreover, among the reference positions, the second imagecapture conditions are applied to the image data item Cr2. Due to this,the image processing unit 33 (i.e. the correction unit 33 b) performscorrection processing on the image data item Cr2. And subsequently theimage processing unit 33 (i.e. the generation unit 33 c) calculates theimage data of the color difference component Cr at the position forattention (i.e. at the second row and the second column).

Furthermore, in FIG. 9C, the position shown by the thick frame (at thesecond row and the third column) is the position for attention of thecolor difference coefficient Cr. Moreover, the image data items Cr2,Cr4, Cr5, and Cr6 for the color difference component in FIG. 9C arereference positions for performing image processing on the pixel at theposition for attention (at the second row and the third column). In FIG.9C, the second image capture conditions are applied to the position forattention (at the second row and the third column). And, among thereference positions, the first image capture conditions are applied tothe image data items Cr4 and Cr5. Moreover, among the referencepositions, the second image capture conditions are applied to the imagedata items Cr2 and Cr6. Because of this, the image processing unit 33(i.e. the correction unit 33 b) performs correction processing on theimage data items Cr4 and Cr5. And subsequently the image processing unit33 (i.e. the generation unit 33 c) calculates the image data of thecolor difference component Cr at the position for attention (at thesecond row and the third column).

After the image processing unit 33 (i.e. the generation unit 33 c) hasobtained the image data of the color difference component Cr at eachpixel position, it then is able to obtain the image data for the R colorcomponent at each pixel position by adding the image data for the Gcolor component shown in FIG. 8C corresponding to each pixel position.

B Color Interpolation

FIG. 10A is a figure in which the image data of the B color componenthas been extracted from FIG. 8A. The image processing unit 33 (i.e. thegeneration unit 33 c) calculates the image data of the color differencecomponent Cb shown in FIG. 10B on the basis of the image data for the Gcolor component shown in FIG. 8C and the image data for the B colorcomponent shown in FIG. 10A.

First, the usual interpolation of the color difference component Cb willbe explained. When, for example, generating image data for the colordifference component Cb at the position for attention shown by the thickframe (at the third row and the third column) in FIG. 10B, the imageprocessing unit 33 (i.e. the generation unit 33 c) refers to the fouritems of image data Cb1 through Cb4 of the color difference componentpositioned in the neighborhood of the position for attention (at thethird row and the third column). For example, the image processing unit33 (i.e. the generation unit 33 c) may take the value(uCb1+vCb2+wCb3+xCb4)/4 as being the image data for the color differencecomponent Cb at the position for attention (at the third row and thethird column). It should be understood that “u” through “x2 here areweighting coefficients that are provided to correspond to the distancesbetween the reference positions and the position for attention, and tothe structure of the image.

In a similar manner, when, for example, generating image data for thecolor difference component Cb at the position for attention shown by thethick frame in FIG. 10C (at the third row and the fourth column), theimage processing unit 33 (i.e. the generation unit 33 c) refers to thefour items of image data Cb2 and Cb4 through Cb6 of the color differencecomponent positioned in the neighborhood of the position for attention(at the third row and the fourth column). For example, the imageprocessing unit 33 (i.e. the generation unit 33 c) may take the value(yCb2+zCb4+αCb5+βCb6)/4 as being the image data for the color differencecomponent Cb at the position for attention (at the third row and thefourth column). It should be understood that y, z, α, and β here areweighting coefficients that are provided to correspond to the distancesbetween the reference positions and the position for attention, and tothe structure of the image. In this manner, image data for the colordifference component Cb is generated for all the pixel positions.

Next, the interpolation of the color difference component Cb in thisembodiment will be explained. In FIGS. 10A through 10C, for example, itis supposed that the first image capture conditions are applied to theregion that is leftward of and above the thick line, while the secondimage capture conditions are applied to the region that is rightward ofand below the thick line. It should be understood that, in FIGS. 10Athrough 10C, the first image capture conditions and the second imagecapture conditions are different from one another. In FIG. 10B, theposition shown by the thick frame (at the third row and the thirdcolumn) is the position for attention of the color difference componentCb. Moreover, the image data items Cb1 through Cb4 for the colordifference component in FIG. 10B are reference positions for performingimage processing on the pixel at the position for attention (at thethird row and the third column). In FIG. 10B, the second image captureconditions are applied at the position for attention (at the third rowand the third column). And, among the reference positions, the firstimage capture conditions are applied to the image data items Cb1 andCb3. Moreover, among the reference positions, the second image captureconditions are applied to the image data items Cb2 and Cb4. Because ofthis, the image processing unit 33 (i.e. the correction unit 33 b)performs correction processing on the image data items Cb1 and Cb3. Andsubsequently the image processing unit 33 (i.e. the generation unit 33c) calculates the image data of the color difference component Cb at theposition for attention (the third row and the third column).

Furthermore, in FIG. 10C, the position shown by the thick frame (at thethird row and the fourth column) is the position for attention of thecolor difference coefficient Cb. Moreover, the image data items Cb2 andCb4 through Cb6 for the color difference component in FIG. 10C are thereference positions for performing image processing upon the pixel atthe position for attention (at the third row and the fourth column). InFIG. 10C, the second image capture conditions are applied to theposition for attention (at the third row and the fourth column). And thesecond image capture conditions are applied to the image data items Cb2and Cb4 through Cb6 at all the reference positions. Due to this, theimage processing unit 33 (i.e. the generation unit 33 c) calculates theimage data for the color difference component Cb at the position forattention (at the third row and the fourth column) by referring to theimage data items Cb2 and Cb4 through Cb6 at the reference positions,without any correction processing being performed by the imageprocessing unit 33 (i.e. by the correction unit 33 b).

After having obtained the image data of the color difference componentCb at each pixel position, the image processing unit 33 (i.e. thegeneration unit 33 c) is able to obtain the image data for the B colorcomponent at each pixel position by adding the image data shown in FIG.8C for the G color component corresponding to each pixel position.

It should be understood that although, in the “G color interpolation”described above, when for example generating image data for the G colorcomponent at the position for attention shown by the thick frame in FIG.8B (at the second row and the second column), reference is made to thefour image data items G1 through G4 of the G color component that arepositioned in the neighborhood of the position for attention, it wouldalso be acceptable to arrange to vary the number of image data items ofthe G color component with reference to the structure of the image. Forexample, if the image in the neighborhood of the position for attentionhas the similarity in the vertical direction (for example, if it is apattern of vertical stripes), then the interpolation processing may beperformed by using only the image data above and below the position forattention (i.e., only G1 and G2 in FIG. 8B). Moreover, if for examplethe image in the neighborhood of the position for attention has thesimilarity in the horizontal direction (for example, if it is a patternof horizontal stripes), then the interpolation processing may beperformed by using only the image data on the left and the right of theposition for attention (i.e., only G3 and G4 in FIG. 8B). In thesecases, the image data item G4 upon which correction is performed by thecorrection unit 33 b may be, or may not be used.

(3) Contour Emphasis Processing

An example of the contour emphasis processing will now be explained. Inthe image for one frame, for example, the image processing unit 33 (i.e.the generation unit 33 c) may perform a per se known linear filtercalculation by employing a kernel of a predetermined size that iscentered on the pixel for attention P (i.e. on the pixel that is thesubject of processing). If the size of the kernel of the sharpeningfilter, which is an example of a linear filter, is N×N pixels, and theposition of the pixel for attention P is the position for attention,then the positions of the (N²−1) reference pixels surrounding the pixelfor attention P are the reference positions.

It should be understood that it would also acceptable for the size ofthe kernel to be N×M pixels.

The image processing unit 33 (i.e. the generation unit 33 c) performsfilter processing to replace the image data at the pixel for attention Pwith the result of the linear filter calculation, while shifting thepixel for attention from left to right along successive horizontallines, for example from the horizontal line at the upper portion of theframe image toward the horizontal line at the lower portion thereof.

In this embodiment, if a pixel to which second image capture conditionshave been applied that are different from the first image captureconditions that were applied to the pixel for attention P is included inthe reference pixels described above, then the image processing unit 33(i.e. the correction unit 33 b) performs correction processing upon theimage data to which the second image capture conditions have beenapplied. And subsequently the image processing unit 33 (i.e. thegeneration unit 33 c) performs the linear filter processing describedabove.

(4) Noise Reduction Processing

An example of noise reduction processing will now be explained. In theimage for one frame, the image processing unit 33 (i.e. the generationunit 33 c) may perform, for example, a per se known linear filtercalculation by employing a kernel of a predetermined size that iscentered on the pixel for attention P (i.e. on the pixel that is thesubject of processing). If the size of the kernel of the smoothingfilter, which is an example of a linear filter, is N×N pixels, and theposition of the pixel for attention P is the position for attention,then the positions of the (N²−1) reference pixels surrounding the pixelfor attention P are the reference positions.

It should be understood that it would also acceptable for the size ofthe kernel to be N×M pixels.

The image processing unit 33 (i.e. the generation unit 33 c) performsfilter processing to replace the image data at the pixel for attention Pwith the result of linear filter calculation, while shifting the pixelfor attention from left to right along successive horizontal lines, forexample from the horizontal line at the upper portion of the frame imagetoward the horizontal line at the lower portion thereof.

In this embodiment, if a pixel to which the second image captureconditions have been applied that are different from the first imagecapture conditions applied to the pixel for attention P is included inthe above described reference pixels, then the image processing unit 33(i.e. the correction unit 33 b) performs correction processing on theimage data to which the second image capture conditions have beenapplied. And subsequently the image processing unit 33 (i.e. thegeneration unit 33 c) performs the linear filter processing describedabove.

2. When Performing Focus Detection Processing

The control unit 34 (i.e. the AF calculation unit 34 d) performs focusdetection processing by employing signal data (i.e. image data)corresponding to a predetermined position (the focus detection position)in the image capturing screen. If different image capture conditions areset for different ones of the divided regions, and the focus detectionposition for AF operation is positioned at a boundary portion betweenthese subdivided regions, then, as pre-processing for the focusdetection processing, the control unit 34 (i.e. the AF calculation unit34 d) performs correction processing on the signal data for focusdetection from at least one of those regions.

This correction processing is performed in order to suppress thereduction of the accuracy of focus detection processing that mightoriginate due to the image capture conditions being different fordifferent regions in the image capturing screen divided by the settingunit 34 b. For example, if the signal data for focus detection at thefocus detection position at which the amount of image deviation (i.e.the phase difference) in the image is detected is positioned near aboundary between the subdivided regions, then signal data to whichdifferent image capture conditions have been applied may be mixedtogether in the signal data for focus detection. In this embodiment,rather than performing detection of the amount of image deviation (i.e.of the phase difference) by employing the signal data to which differentimage capture conditions have been applied just as it is withoutalteration, correction processing as described below is performed, onthe basis of the consideration that it is more desirable to performdetection of the amount of image deviation (i.e. of the phasedifference) by employing signal data on which correction processing hasbeen performed, so as to suppress any discrepancy in the signal data dueto variation of the image capture conditions.

Example of Focus Detection Processing

An example of such focus detection processing accompanied by correctionprocessing will now be described. In the AF operation according to thisembodiment, for example, the focus is adjusted to a photographic subjectthat corresponds to a focus detection position that is selected by theuser from among a plurality of focusing points in the image capturingscreen. The control unit 34 (i.e. the AF calculation unit 34 d (thegeneration unit)) calculates the amount of defocusing of the imagecapturing optical system 31 by detecting the amount of image deviation(i.e. the phase difference) of a plurality of images of the photographicsubject formed by the light fluxes having passed through different pupilregions of the image capturing optical system 31. And the control unit34 (i.e. the AF calculation unit 34 d) shifts a focusing lens of theimage capturing optical system 31 to a position that makes the amount ofdefocusing become zero (or less than some permitted value), in otherwords to its focusing position.

FIG. 11 is a figure showing an example of positioning of pixels forfocus detection in the image capturing surface of the imaging element 32a. In this embodiment, pixels for focus detection are providedseparately as lined up along the X axis direction (i.e. the horizontaldirection) of the image capturing chip 111. In the example of FIG. 11 ,fifteen focus detection pixel lines 160 are provided, spaced apart atpredetermined intervals. Each of the pixels for focus detection thatmake up these focus detection pixel lines 160 outputs aphotoelectrically converted signal for focus detection. Normal pixelsfor image capturing are provided in the image capturing chip 111 atpositions other than those of the focus detection pixel lines 160. Thesepixels for image capturing output photoelectrically converted signalsfor provision of a live view image, and/or for recording.

FIG. 12 is a figure giving an enlarged view of a partial region of oneof the focus detection pixel lines 160 described above corresponding toa focus detection position 80A shown in FIG. 11 . In FIG. 12 , examplesare shown of red color pixels R, green color pixels G (Gb and Gr), bluecolor pixels B, pixels for focus detection S1, and other pixels forfocus detection S2. The red color pixels R, the green color pixels G (Gband Gr), and the blue color pixels B are arranged according to the ruleof the Bayer array described above.

The square shaped regions shown by way of example for the red colorpixels R, the green color pixels G (Gb and Gr), and the blue colorpixels B represent the light receiving areas of these image capturingpixels. Each of the pixels for image capturing receives the light fluxthat has passed through an exit pupil of the image capturing opticalsystem 31 (refer to FIG. 1 ). In other words, each of the red colorpixels R, the green color pixels G (Gb and Gr), and the blue colorpixels B has a square shaped mask opening portion, and light passingthrough these mask opening portions reaches the light reception portionsof these image capturing pixels.

It should be understood that the shapes of the light reception regions(i.e. of the mask openings) of the red color pixels R, the green colorpixels G (Gb and Gr), and the blue color pixels B are not limited tobeing quadrilateral; for example, they could be circular.

The semicircular shaped regions shown by way of example for the pixelsfor focus detection S1 and the pixels for focus detection S2 representthe light receiving areas of these pixels for focus detection. In otherwords, each of the pixels for focus detection S1 has a semicircularshaped mask opening portion on the left side of its pixel position inFIG. 12 , and light passing through this mask opening portion reachesthe light reception portion of this pixel for focus detection SL. On theother hand, each of the pixels for focus detection S2 has a semicircularshaped mask opening portion on the right side of its pixel position inFIG. 12 , and light passing through this mask opening portion reachesthe light reception portion of this pixel for focus detection S2. Inthis manner, the pixels for focus detection S1 and the pixels for focusdetection S2 respectively receive one of a pair of light fluxes thathave passed through different regions of the exit pupil of the imagecapturing optical system 31 (refer to FIG. 1 ).

It should be understood that the positions of the focus detection pixellines 160 in the image capturing chip 111 are not limited to being thepositions shown by way of example in FIG. 11 . Moreover, the number ofthe focus detection pixel lines 160 is not limited to the number thereofshown by way of example in FIG. 11 . Furthermore, the shapes of the maskopening portions in the pixels for focus detection S1 and in the pixelsfor focus detection S2 are not limited to being semicircular; forexample, it would be acceptable to arrange to divide the quadrilaterallight reception regions (i.e. the mask opening portions) of the imagecapturing pixels R, the image capturing pixels G, and the imagecapturing pixels B in the horizontal direction, so as to definerectangular shapes.

Yet further, it would also be acceptable for the focus detection pixellines 160 in the image capturing chip 111 to be lines in which thepixels for focus detection are provided as lined up along the Y axisdirection of the image capturing chip 111 (i.e. along the verticaldirection). Even further, an imaging element of the type shown in FIG.12 in which pixels for image capturing and pixels for focus detectionare arranged in a two dimensional array is per se known, and accordinglythe details thereof are not shown in the figures, and explanationthereof will be curtailed.

It should be understood that, in the FIG. 12 example, a so-called 1PDconfiguration has been explained in which each of the pixels S1 and S2for focus detection receives one of a pair of light fluxes for focusdetection. Instead of this, it would also be acceptable to arrange toprovide a so-called 2PD configuration in which each of the pixels forfocus detection receives both of the pair of light fluxes for focusdetection. By adopting the 2PD configuration, it becomes possible alsoto employ the photoelectrically converted signals obtained from thepixels for focus detection as photoelectrically converted signals forrecording.

On the basis of the photoelectrically converted signals for focusdetection outputted from the pixels for focus detection S1 and from thepixels for focus detection S2, the control unit 34 (i.e. the AFcalculation unit 34 d) detects the amount of image deviation (i.e. thephase difference) between the pair of images formed by the pair of lightfluxes that have passed through different regions of the image capturingoptical system 31 (refer to FIG. 1 ). And the amount of defocusing iscalculated on the basis of the amount of image deviation (i.e. the phasedifference). Since calculation of the amount of defocusing according tothis type of split pupil phase method is per se known in the camerafield, accordingly detailed description thereof will be omitted.

It will be supposed that the focus detection position 80A (refer to FIG.11 ) is selected by the user to a position in the live view image 60 ashown by way of example in FIG. 7A that corresponds to a region 80 nearthe boundary of the region 61. FIG. 13 is an enlarged view showing thefocus detection position 80A. Pixels with white background indicate thatthe first image capture conditions are set, while shaded pixels indicatethat the second image capture conditions are set. In FIG. 13 , theposition surrounded by the frame 170 corresponds to one of the focusdetection pixel lines 160 (refer to FIG. 11 ).

Normally, the control unit 34 (i.e. the AF calculation unit 34 d)performs focus detection processing by employing the signal data fromthe pixels for focus detection shown by the frame 170 just as it iswithout alteration. However, if signal data to which the first imagecapture conditions have been applied and signal data to which the secondimage capture conditions have been applied are mixed together in thesignal data surrounded by the frame 170, then, among the signal datasurrounded by the frame 170, the control unit 34 (i.e. the AFcalculation unit 34 d) performs correction processing on the signal datato which the second image capture conditions have been applied, as shownin the following Example #1 through Example #3. And the control unit 34(i.e. the AF calculation unit 34 d) performs focus detection processingby utilizing the signal data after this correction processing.

Example #1

If, for example, the only difference between the first image captureconditions and the second image capture conditions is the ISOsensitivity, with the ISO sensitivity in the first image captureconditions being 100 while the ISO sensitivity in the second imagecapture conditions is 800, then, as correction processing, the controlunit 34 (i.e. the AF calculation unit 34 d) multiplies the signal datacaptured under the second image capture conditions by 100/800. By doingthis, variance in the signal data due to the difference between theimage capture conditions is reduced.

It should be understood that, although variance in the signal databecomes small if the amounts of light incident on the pixels to whichthe first image capture conditions are applied and the amounts of lightincident on the pixels to which the second image capture conditions areapplied are the same, by contrast, in principle, if the amounts of lightincident upon the pixels to which the first image capture conditions areapplied and the amounts of light incident upon the pixels to which thesecond image capture conditions are applied are different, or the like,then the variance in the signal data may not become small. The same istrue for the other examples described below.

Example #2

If, for example, the only difference between the first image captureconditions and the second image capture conditions is the shutter speed,with the shutter speed in the first image capture conditions being1/1000 sec. while the shutter speed in the second image captureconditions is 1/100 sec., then, as correction processing, the controlunit 34 (i.e. the AF calculation unit 34 d) multiplies the signal datacaptured under the second image capture conditions by 1/1000/ 1/100=1/10. By doing this, variance in the signal data caused by the differentimage capture conditions is reduced.

Example #3

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their frame rate(and with the charge accumulation time being the same), with the framerate in the first image capture conditions being 30 fps while the framerate in the second image capture conditions is 60 fps, then, ascorrection processing, the control unit 34 (i.e. the AF calculation unit34 d) employs, for the signal data that was acquired under the secondimage capture conditions (i.e. at 60 fps), the signal data for thoseframe images whose starting timings of acquisition are close to those ofthe frame images that was acquired under the first image captureconditions (i.e. at 30 fps). By doing this, variance in the signal datadue to difference in the image capture conditions is reduced.

It should be understood that it would also be acceptable to perform thecorrection processing by performing interpolation calculation of thesignal data for the frame images whose starting timings of acquisitionare close to those of the frame images that were acquired under thefirst image capture conditions (at 30 fps), on the basis of a pluralityof the previous and subsequent frame images that were acquired under thesecond image capture conditions (60 fps).

On the other hand, if the image capture conditions that were applied tothe image data surrounded by the frame 170 are all the same, then itwill be acceptable for the control unit 34 (i.e. the AF calculation unit34 d) not to perform any correction processing such as described above.In other words, the control unit 34 (i.e. the AF calculation unit 34 d)performs focus detection processing by employing the signal data fromthe pixels for focus detection shown by the frame 170 just as it iswithout alteration.

It should be understood that, as described above, even if there are somerelatively insubstantial differences in the image capture conditions,they are nevertheless considered to be the same image captureconditions.

Furthermore although, in the example described above, a case has beenexplained in which correction processing is performed upon, among thesignal data, the signal data that was captured under the second imagecapture conditions by referring to the first image capture conditions,it would also be acceptable to perform correction processing upon, amongthe signal data, the signal data that was captured under the first imagecapture conditions by referring to the second image capture conditions.

It would also be acceptable to arrange for the control unit 34 (i.e. theAF calculation unit 34 d) to decide whether to perform correctionprocessing on the signal data that was acquired under the first imagecapture conditions, or to perform correction processing upon the signaldata that was acquired under the second image capture conditions, forexample on the basis of the ISO sensitivity or the like. If the firstimage capture conditions and the second image capture conditions differin their ISO sensitivities, then it is desirable to perform correctionprocessing on the signal data that was acquired under the image captureconditions whose ISO sensitivity is the lower, provided that the signaldata that was acquired under the image capture conditions whose ISOsensitivity is the higher is not saturated. In other words, if the firstimage capture conditions and the second image capture conditions differin their ISO sensitivities, then it is more desirable to performcorrection processing on the signal data that is darker, so as to reduceits difference from the signal data that is brighter.

Even further, it would also be acceptable to arrange to reduce thedifference after the correction processing between the signal data thathas been acquired under the first image capture conditions and thesignal data that has been acquired under the second image captureconditions, by performing correction processing upon both those sets ofsignal data.

In the above explanation, as an example, focus detection processingemploying the split pupil phase method has been disclosed, but it wouldalso be possible to perform focus detection processing in a similarmanner in the case of employing a contrast detection method in which thefocusing lens of the image capturing optical system 31 is shifted to itsfocusing position on the basis of the magnitude of the contrast of thephotographic subject image.

In the case of employing such a contrast detection method, whileshifting the focusing lens of the image capturing optical system 31, ateach position of the focusing lens, the control unit 34 performs per seknown calculation of a focus evaluation value on the basis of the signaldata outputted from the image capturing pixels of the imaging element 32a corresponding to the focus detection position. And then the positionof the focusing lens that makes the focus evaluation value attain itsmaximum is taken as being its focusing position.

Normally, the control unit 34 performs calculation of the focusevaluation value by employing the signal data outputted from the imagecapturing pixels corresponding to the focus detection position withoutperforming any correction processing. However, if signal data to whichthe first image capture conditions have been applied and signal data towhich the second image capture conditions have been applied are mixedtogether in the signal data corresponding to the focus detectionposition, then, among the signal data corresponding to the focusdetection position, the control unit 34 performs correction processingas described above upon the signal data that was acquired under thesecond image capture conditions. And the control unit 34 performscalculation of the focus evaluation value by employing the signal dataafter this correction processing.

3. When Performing Photographic Subject Detection Processing

FIG. 14A is a figure showing an example of a template image representingan object that is to be detected, and FIG. 14B is a figure showingexamples of a live view image 60(a) and a search range 190. The controlunit 34 (i.e. the object detection unit 34 a) detects the object fromthe live view image (in this example, the bag 63 a, which is one of theelements of the photographic subject of FIG. 5 ). It will be acceptableto arrange for the control unit 34 (i.e. the object detection unit 34 a)to take the range for detection of the object as being the entire rangeof the live view image 60 a, but, in order to reduce the burden of thedetection processing, it would also be acceptable to take only a portionof the live view image 60 a as being the search range 190.

In the case in which different image conditions are set for thedifferent divided regions, and the search range 190 includes a boundarybetween the divided regions, as pre-processing for the photographicsubject detection processing, the control unit 34 (i.e. the objectdetection unit 34 a) performs correction processing on the image datafor at least one of the regions within the search range 190.

Correction processing is performed in order to suppress reduction of theaccuracy of the photographic subject element detection processing,caused by the different image capture conditions in the regions intowhich the image capturing screen has been subdivided by the setting unit34 b. In general, if the search range 190 that is employed for detectionof the photographic subject elements includes a boundary between two ofthe subdivided regions, then image data to which different image captureconditions have been applied may be mixed together in the image data forthe search range 190. In this embodiment, rather than performingdetection of the elements of the photographic subject by employing suchimage data to which different image capture conditions have been appliedjust as they are without alteration, correction processing is performedas described below, on the basis of the consideration that it is moredesirable to perform detection of the photographic subject elements byemploying image data upon which correction processing has been performedso as to suppress variance in the image data due caused by the differentimage capture conditions.

For the live view image 60 a shown by way of example in FIG. 5 , thecase will now be explained of detecting the bag 63 a, which is an objectthat is being held by the person 61 a. The control unit 34 (i.e. theobject detection unit 34 a) sets the search range 190 to theneighborhood of the region that includes the person 61 a. It should beunderstood that it would also be acceptable to set the region 61 thatincludes the person 61 a as the search range.

In the case in which the search range 190 is not divided up into tworegions for which the image capture conditions are different, thecontrol unit 34 (i.e. the object detection unit 34 a) performs thephotographic subject detection processing by employing the image datarepresenting the search range 190 without performing any correctionprocessing. However, supposing that the image data for which first imagecapture conditions have been applied and the image data for which secondimage capture conditions have been applied are mixed together in theimage data for the search range 190, then the control unit 34 (i.e. theobject detection unit 34 a) performs correction processing on, among theimage data for the search range 190, the image data that was acquiredunder the second image capture conditions, as described in Examples #1through #3 in the case of performing focus detection processing. And thecontrol unit 34 (i.e. the object detection unit 34 a) performsphotographic subject detection processing by employing the image dataafter this correction processing.

It should be understood that, as described above, even if there are somerelatively insubstantial differences in the image capture conditions,they are nevertheless considered to be the same image captureconditions.

Furthermore, although in the example described above a case has beenexplained in which correction processing is performed according to thefirst image capture conditions upon, among the image data, the imagedata that was captured under the second image capture conditions, itwould also be acceptable to perform correction processing according tothe second image capture conditions upon, among the image data, theimage data that was captured under the first image capture conditions.

It will also be acceptable to apply the correction processing for theimage data for the search range 190 described above to a search rangethat is employed in order to detect a specific photographic subject suchas the face of a person or the like, or to a region that is employed fordetermining the type of scene that has been photographed.

Furthermore, the above described correction processing for the imagedata for the search range 190 is not limited to being applied to asearch range that is employed in a pattern matching method that employsa template image, but may also be applied, in a similar manner, to asearch range for detection of the amount of some characteristic of theimage, on the basis of its color or its contour or the like.

Yet further it would also be acceptable, by performing per se knowntemplate matching processing employing image data for a plurality offrames whose time points of acquisition are different, to apply a regionthat is similar to a tracking subject in a frame image that is firstacquired, to tracking processing of a moving object that is found fromframe images that are subsequently acquired. In this case, if image datato which first image capture conditions have been applied and image datato which second image capture conditions have been applied are mixedtogether in the search ranges that are set for the frame images that aresubsequently acquired, then the control unit 34 performs correctionprocessing as described in Examples #1 through #3 above upon, among theimage data for the search range, the image data that was acquired underthe second image capture conditions. And then the control unit 34performs tracking processing by employing the image data after thiscorrection processing.

Even further, the same also applies to a known method for detecting amovement vector by employing image data for a plurality of frames whosetime points of acquisition are different. If image data to which firstimage capture conditions have been applied and image data to whichsecond image capture conditions have been applied are mixed together inthe detection region that is employed for detection of the movementvector, then the control unit 34 performs correction processing asdescribed in Examples #1 through #3 above upon, among the image data forthe detection region that is employed for detection of the movementvector, the image data that was acquired under the second image captureconditions. And the control unit 34 detects the movement vector byemploying the image data after this correction processing.

4. When Setting Image Capture Conditions

In the state in which the image capturing screen is subdivided intoregions and different image capture conditions are set for differentones of these subdivided regions, if the control unit 34 (i.e. thesetting unit 34 b) newly performs photometry and determines new exposureconditions again, then correction processing is performed upon the imagedata for at least one of the regions, as pre-processing to set itsexposure conditions.

The correction processing is performed in order to suppressdeterioration of the accuracy of the processing for determination of theexposure conditions, caused by difference of the image captureconditions between the different regions into which the image capturingscreen has been divided by the setting unit 34 b. For example, if aboundary between the divided regions is included in a photometric rangethat is set in the central portion of the image capturing screen, thenimage data to which different image capture conditions have been appliedmay be present in the image data for the photometric range. In thepresent embodiment, rather than performing the exposure calculationprocessing by employing the image data for which different image captureconditions have been applied just as it is without alteration,correction processing as described below is performed on the basis ofthe consideration that it is more desirable to perform the exposurecalculation processing by employing the image data upon which correctionprocessing has been performed in order to suppress variance in the imagedata due to difference in the image capture conditions.

If the photometric range is not subdivided by a plurality of regions forwhich the image capture conditions are different, then the control unit34 (i.e. the setting unit 34 b) performs the exposure calculationprocessing by employing the image data making up the photometric rangejust as it is without performing any correction processing. However,supposing that image data to which first image capture conditions havebeen applied and image data to which second image capture conditionshave been applied are mixed together in the image data for thephotometric range, then the control unit 34 (i.e. the setting unit 34 b)performs correction processing upon the image data, among the image datafor the photometric range, to which the second image capture conditionswere applied, as described in Example #1 through Example #3 above forthe cases of performing the focus detection processing and thephotographic subject detection processing. And then the control unit 34(i.e. the setting unit 34 b) performs exposure calculation processing byemploying the image data after this correction processing.

It should be understood that, as described above, even if there are somerelatively insubstantial differences in the image capture conditions,they are nevertheless considered to be the same image captureconditions.

Furthermore although, in the example described above, a case has beenexplained in which correction processing is performed according to thefirst image capture conditions upon, among the image data, the imagedata that was captured under the second image capture conditions, itwould also be acceptable to perform correction processing according tothe second image capture conditions upon, among the image data, theimage data that was captured under the first image capture conditions.

The above considerations are not only limited to the photometric rangewhen performing the exposure calculation processing described above; thesame holds for the photometric (colorimetry) range that is employed whendetermining the white balance adjustment value, for the photometricrange that is employed when determining whether or not it is proper toemit auxiliary photographic light from a light source that emitsauxiliary photographic light, and to the photometric range that isemployed when determining the amount of light to be emitted as auxiliaryphotographic light from the light source described above.

Furthermore, if the resolutions at which the photoelectric conversionsignals are read out are to be made different between the differentregions into which the image capturing screen is divided, the similartechnique may be applied to regions that are employed for determiningthe type of the image capturing scene which are used to determine theresolution at which each region is to be read out.

Explanation of Flow Chart

FIG. 15 is a flow chart for explanation of the flow of processing forsetting image capture conditions for each region individually andperforming image capturing processing. When a main switch of the camera1 is turned on, the control unit 34 starts a program for executing theprocessing shown in FIG. 15 . In step S10, the control unit 34 starts toprovide a live view display upon the display unit 35, and then the flowof control proceeds to step S20.

In concrete terms, the control unit 34 commands the imaging unit 32 tostart acquisition of live view images, and causes the live view imagesthat are thus acquired to be successively displayed upon the displayunit 35. As described above, at this time point the same image captureconditions are set for the entire area of the image capturing chip 111,in other words for the entire screen.

It should be understood that, if a setting is made for performing AFoperation during this live view display, then, by performing focusdetection processing, the control unit 34 (i.e. the AF calculation unit34 d) controls AF operation to adjust the focus to the element of thephotographic subject that corresponds to a predetermined focus detectionposition. And, the AF calculation unit 34 d performs focus detectionprocessing after having performed the correction processing describedabove as necessary.

Furthermore, if no setting is made for performing AF operation duringthe live view display, then the control unit 34 (i.e. the AF calculationunit 34 d) performs AF operation when AF operation is subsequentlycommanded.

In step S20, the control unit 34 (i.e. the object detection unit 34 a)detects one or more elements in the photographic subject from the liveview image, and then the flow of control proceeds to step S30. And,after having performed the correction processing described above ifnecessary, the object detection unit 34 a performs photographic subjectdetection processing. In step S30, the control unit 34 (i.e. the settingunit 34 b) subdivides the screen of the live view image into regionscontaining different elements of the photographic subject, and then theflow of control proceeds to step S40.

In step S40, the control unit 34 performs display of the regions uponthe display unit 35. And, as shown in FIG. 6 , among the subdividedregions, the control unit 34 displays the region that is the subject ofsetting (or of changing) its image capture conditions as accentuated.Furthermore, the control unit 34 causes the display unit 35 to displaythe image capture conditions setting screen 70, and then the flow ofcontrol proceeds to step S50.

It should be understood that, if the display position of some other mainphotographic subject is tapped on the display screen by the finger ofthe user, then the control unit 34 changes the region that includes thisselected main photographic subject to newly become the region that isthe subject of setting (or of changing) its image capture conditions,and causes this region to be displayed as accentuated.

In step S50, the control unit 34 makes a decision as to whether or notAF operation is required. If, for example, the focal point adjustmentstate has changed due to movement of the photographic subject, or if theposition of the point for focus detection has changed due to useractuation, or if due to user actuation a command has been issued for AFoperation to be performed, then the control unit 34 reaches anaffirmative decision in step S50 and the flow of control is transferredto step S70. But if the focal point adjustment state has not changed,and the position of the point for focus detection has not changed due touser actuation, and no command has been issued due to user actuation forAF operation to be performed, then the control unit 34 reaches anegative decision in step S50, and the flow of control is transferred tostep S60.

In step S70, the control unit 34 causes AF operation to be performed,and then the flow of control returns to step S40. According torequirements, the AF calculation unit 34 d performs the focus detectionprocessing, i.e. the AF operation, after having performed the correctionprocessing described above. And, after the flow of control has returnedto step S40, the control unit 34 repeats processing similar to thatdescribed above on the basis of the live view image that is acquiredafter the AF operation.

On the other hand, in step S60, according to user actuation, the controlunit 34 (i.e. the setting unit 34 b) sets the image capture conditionsfor the region that is being displayed as accentuated, and then the flowof control is transferred to step S80. It should be understood that thechanging over of the display of the display unit 35 and the setting ofimage capture conditions according to user actuation in the step S60,are as described above. And, according to requirements, the control unit34 (i.e. the setting unit 34 b) performs exposure calculation processingafter having performed the correction processing described above.

In step S80, the control unit 34 makes a decision as to whether or notan image capture command has been issued. If a release button not shownin the figures included in the actuation members 36 has been actuated,or a display icon for commanding image capture has been actuated, thenthe control unit 34 reaches an affirmative decision in step S80 and theflow of control proceeds to step S90. On the other hand, if no imagecapture command has been issued, then the control unit 34 reaches anegative decision in step S80 and the flow of control returns to stepS60.

In step S90, the control unit 34 performs predetermined image captureprocessing. That is, the image capturing control unit 34 c control theimage capturing element 32 a to perform image capture under the imagecapture conditions that have been set for each region as describedabove, and then the flow of control proceeds to step S100.

In step S100, the control unit 34 (i.e. the imaging control unit 34 c)sends a command to the image processing unit 33, and thereby causespredetermined image processing to be performed upon the image dataobtained by the image capture described above, and then the flow ofcontrol proceeds to step S110. This image processing may include thepixel defect correction processing, the color interpolation processing,the contour emphasis processing, and the noise reduction processingdescribed above.

It should be understood that, according to requirements, the imageprocessing unit 33 (i.e. the correction unit 33 b) may perform the imageprocessing after having performed correction processing upon the imagedata that is positioned in the vicinity of a boundary between regions.

In step S110, the control unit 34 sends a command to the recording unit37 and causes it to record the image data after image processing uponthe recording medium not shown in the figures, and then the flow ofcontrol proceeds to step S120.

In step S120, the control unit 34 determines whether or not atermination actuation has been performed. If a termination actuation hasbeen performed, then the control unit 34 reaches an affirmative decisionin step S120 and the processing of FIG. 15 terminates. But if notermination actuation has been performed, then the control unit 34reaches a negative decision in step S120 and the flow of control returnsto step S20. If the flow of control returns to step S20, then thecontrol unit 34 repeats the processing described above.

While, in the explanation given above, an example has been given inwhich a laminated type imaging element 100 is employed as the imagingelement 32 a, it is not necessary for the imaging element to be of thelaminated type, provided that it is capable of setting image captureconditions individually for each of a plurality of blocks upon theimaging element (i.e. upon the image capturing chip 111).

According to the first embodiment of the present invention describedabove, the following advantageous operational effects are obtained.

(1) This camera 1 equipped with an image processing device comprises thecontrol unit 34 (i.e. the setting unit 34 b) that sets image captureconditions for the first region of the imaging unit 32 that aredifferent from the image capture conditions for the second region of theimaging unit 32, the image processing unit 33 (i.e. the correction unit33 b) that corrects the first image data generated by image capture oflight incident upon the first region under the image capturingconditions set by the setting unit 34 b on the basis of the imagecapture conditions for the second region set by the setting unit 34 b,and the image processing unit 33 (i.e. the generation unit 33 c) thatgenerates an image by the first image data corrected by the correctionunit 33 b and the second image data generated by image capture of lightincident upon the second region. Due to this, it is possible to performprocessing in an appropriate manner for each of the regions havingdifferent image capture conditions. In other words, it is possible togenerate an image in an appropriate manner by the image data that hasbeen generated for each of the regions. For example, it is possible tosuppress any discontinuity or strangeness appearing in the image thatmight be generated due to differences in the image capture conditionsbetween the various regions.

(2) Since the correction unit 33 b of the camera 1 corrects the firstimage data so that the differences between the values of the first imagedata and the values of the second image data are reduced, accordingly itis possible to perform processing in an appropriate manner for both ofthese two regions whose image capture conditions are different.

(3) Since the correction unit 33 b of the camera 1 corrects the firstimage data on the basis of variance between the image capture conditionsfor the first region and the image capture conditions for the secondregion, accordingly it is possible to perform processing in anappropriate manner for each of the regions whose image captureconditions are different.

(4) It would also be acceptable to arrange for the correction unit 33 bof the camera 1 to correct the second image data on the basis of theimage capture conditions set for the first region by the setting unit 34b, and for the generation unit 33 c to generate an image from the firstimage data that has been corrected by the correction unit 33 b and fromthe second image data that has also been corrected by the correctionunit 33 b. By doing this, again, it is possible to perform processing inan appropriate manner for each of the regions whose image captureconditions are different.

(5) It would also be acceptable to arrange for the correction unit 33 bof the camera 1 to correct the second image data according to the firstimage data, and for the generation unit 33 c to generate an imageaccording to the first image data that has been corrected by thecorrection unit 33 b and the second image data that has also beencorrected by the correction unit 33 b. Also, by doing this, it ispossible to perform processing in an appropriate manner for each of theregions whose image capture conditions are different.

(6) Since the generation unit 33 c of the camera 1 performs imageprocessing upon the second image data by employing the first image datathat has been corrected by the correction unit 33 b, accordingly it ispossible to perform processing in an appropriate manner for each of theregions whose image capture conditions are different.

(7) Since the generation unit 33 c of the camera 1 changes the value ofthe second image data by employing the value of the first image datathat have been corrected by the correction unit 33 b, accordingly it ispossible to perform processing in an appropriate manner for each of theregions whose image capture conditions are different.

(8) Since the generation unit 33 c of the camera 1 replaces the value ofthe second image data with the value of the first image data that havebeen corrected by the correction unit 33 b, accordingly it is possibleto perform processing in an appropriate manner for each of the regionswhose image capture conditions are different.

(9) Since the generation unit of the camera 1 generates third image datafrom the second image data and the first image data that has beencorrected by the correction unit 33 b, accordingly it is possible toperform processing in an appropriate manner for each of the regionswhose image capture conditions are different.

(10) The setting unit 34 b of the camera 1 sets the image capturesensitivity for the first region of the imaging unit 32 to a first imagecapture sensitivity as the image capture conditions for the firstregion, and sets the image capture sensitivity for the second region ofthe imaging unit 32 to a second image capture sensitivity that isdifferent from the first image capture sensitivity as the image captureconditions for the second region, and the correction unit 33 b correctsthe first image data that has been generated by image capture under theimage capture conditions set by the setting unit 34 b on the basis ofthe difference between the first image capture sensitivity and thesecond image capture sensitivity. Due to this, it is possible to performprocessing in an appropriate manner for each of the regions whose imagecapturing sensitivities are different.

It would also be acceptable to change over between a first mode in whichcorrection processing described above is performed as pre-processing anda second mode in which no correction processing is performed aspre-processing. When the first mode is selected, the control unit 34performs processing such as image processing or the like afterperforming the pre-processing described above. On the other hand, whenthe second mode is selected, the control unit 34 performs processingsuch as image processing or the like without performing thepre-processing described above. For example, if a part of a face thathas been detected as a photographic subject element is in shadow, whencolor interpolation processing is performed after having performedcorrection processing upon an image that has been generated by imagecapture with the image capture conditions for a region that includes theshadowed portion of the face and the image capture conditions for aregion that includes a portion of the face that is not in shadow beingset differently, so that the brightness of the shadowed portion of theface is approximately the same as the brightness of the portion of theface that is not in shadow, then, due to differences in the imagecapture conditions that are set, unintended color interpolation may beperformed upon the face portion that is in shadow. However, by making itpossible to change over between the first mode and the second mode, itbecomes possible to avoid the occurrence of unintended colorinterpolation by enabling color interpolation processing to be performedwhile employing the captured image data just as it is without anycorrection processing being performed thereupon.

———Variants of First Embodiment———

The following modifications also come within the scope of the presentinvention, and moreover it would be possible to combine one or aplurality of these modifications with the embodiment described above.

Variant Embodiment #1

FIG. 16A through 16C are figures showing various arrangements of thefirst region and the second region upon the image capturing surface ofthe imaging element 32 a. According to the example shown in FIG. 16A,the first region consists of a group of even numbered columns, and thesecond region consists of a group of odd numbered columns. In otherwords, the image capturing surface is subdivided into the even numberedcolumns and the odd numbered columns.

And, according to the example shown in FIG. 16B, the first regionconsists of the even numbered rows, and the second region consists ofthe odd numbered rows. In other words, the image capturing surface issubdivided into the even numbered rows and the odd numbered rows.

Moreover, according to the example shown in FIG. 16C, the first regionconsists of the blocks on the even numbered rows in the odd numberedcolumns and the blocks on the odd numbered rows in the even numberedcolumns. And the second region consists of the blocks on the evennumbered rows in the even numbered columns and the blocks on the oddnumbered rows in the odd numbered columns. In other words, the imagecapturing surface is subdivided into a checkerboard pattern.

In each of FIG. 16A through FIG. 16C, a first image that is based uponthe photoelectrically converted signals outputted from the first regionand a second image that is based upon the photoelectrically convertedsignals outputted from the second region are generated from thephotoelectrically converted signals read out from the imaging element 32a after capturing one frame. According to this Variant Embodiment #1,the first image and the second image are captured at the same angle ofview, and include a common photographic subject image.

In this Variant Embodiment #1 of the present invention, the control unit34 employs the first image for display, while employing the second imagefor detection. In concrete terms, the control unit 34 causes the displayunit 35 to display the first image as a live view image. Furthermore,the control unit 34 causes the photographic subject detection processingto be performed by the object detection unit 34 a by employing thesecond image, causes the focus detection processing to be performed bythe AF calculation unit 34 d by employing the second image, and causesthe exposure calculation processing to be performed by the setting unit34 b by employing the second image.

In this Variant Embodiment #1, it will be supposed that the imagecapture conditions that are set for the first region in which the firstimage is captured are termed the “first image capture conditions”, andthat the image capture conditions that are set for the second region inwhich the second image is captured are termed the “second image captureconditions”. The control unit 34 may make the first image captureconditions be different from the second image capture conditions.

1. As one example, the control unit 34 may set the first image captureconditions to be conditions that are appropriate for display by thedisplay unit 35. For example, these first image capture conditions thatare set for the first region may be set to be the same over the entirefirst region of the image capturing screen. On the other hand, thecontrol unit 34 may set the second image capture conditions that are setfor the second region to be conditions that are appropriate for thefocus detection processing, for the photographic subject detectionprocessing, and for the exposure calculation processing. These secondimage capture conditions are set to be the same over the entire secondregion of the image capturing screen.

It should be understood that, if the conditions that are suitable forthe focus detection processing, for the photographic subject detectionprocessing, and for the exposure calculation processing are differentfrom one another, then it would be acceptable for the control unit 34 tomake the second image capture conditions that are set for the secondregion be different for each frame. For example, the second imagecapture conditions for a first frame may be set to be suitable for thefocus detection processing, the second image capture conditions for thenext second frame may be set to be are suitable for the photographicsubject detection processing, and the second image capture conditionsfor the next third frame may be set to be suitable for the exposurecalculation processing. In these cases, for each frame, the second imagecapture conditions are made to be the same over the entire second regionof the image capturing screen.

2. As another example, it would also be acceptable for the control unit34 to make the first image capture conditions that are set for the firstregion vary between different regions. Thus, the control unit 34 (i.e.the setting unit 34 b) sets different first image capture conditions foreach of the regions including photographic subject elements that havebeen subdivided by the setting unit 34 b. On the other hand, the controlunit 34 sets the second image capture conditions that are set for thesecond region to be the same over the entire second region upon theimage capturing screen. The control unit 34 sets the second imagecapture conditions to conditions that are appropriate for the focusdetection processing, for the photographic subject detection processing,and for the exposure calculation processing, but if the conditions thatare respectively appropriate for the focus detection processing, for thephotographic subject detection processing, and for the exposurecalculation processing are different from one another, then it will beacceptable for the image capture conditions set for the second region tobe different for each frame.

3. Moreover, as yet another example, it will also be acceptable for thecontrol unit 34 to set the first image capture conditions that are setfor the first region to be the same over the entire first region of theimage capturing screen, while on the other hand making the second imagecapture conditions that are set for the second region vary betweendifferent regions of the image capturing screen. For example, the secondimage capture conditions may be set to be different for each of theregions including a photographic subject element into which the settingunit 34 b has subdivided the image capturing screen. Also, in this case,if the conditions that are appropriate for the focus detectionprocessing, for the photographic subject detection processing, and forthe exposure calculation processing are different from one another, thenthe image capture conditions that are set for the second region may bedifferent for each frame.

4. Still further, as yet another example, the control unit 34 may makethe first image capture conditions that are set for the first regionvary between different regions of the image capturing screen, and mayalso make the second image capture conditions that are set for thesecond region vary between different regions of the image capturingscreen. For example, along with the first image capture conditions beingset to be different for each of the regions including a photographicsubject element into which the setting unit 34 b has subdivided theimage capturing screen, also the second image capture conditions may beset to be different for each of the regions including a photographicsubject element into which the setting unit 34 b has subdivided theimage capturing screen.

In FIG. 16A through FIG. 16C, it would also be acceptable for the arearatios to be different between the first region and the second region.For example, on the basis of actuation by the user or a determination bythe control unit 34, the control unit 34 may set the area ratio of thefirst region to be higher than that of the second region, or may set thearea ratio of the first region and the area ratio of the second regionto be equal as shown by way of example in FIGS. 16A through 16C, or mayset the area ratio of the first region to be lower than that of thesecond region. By making the area ratios be different between the firstregion and the second region, it is possible to make the resolution ofthe first image be higher as compared with the second image, or to makethe resolutions of the first image and the second image be equal, or tomake the resolution of the second image be higher as compared with thefirst image.

Variant Embodiment #2

In the correction processing when performing image processing, if theimage capture conditions that are applied at the position for attention(here supposed to be the first image capture conditions) and the imageconditions that are applied to one or more reference positions in theneighborhood of the position for attention (here supposed to be thesecond image capture conditions) are different from one another, thenthe image processing unit 33 (i.e. the correction unit 33 b) correctsthe image data captured under the second image capture conditions (i.e.the image data, among the image data for the reference positions, whichwas captured under the second image capture conditions) on the basis ofthe first image capture conditions. In other words, by performingcorrection processing on the image data for the reference position orpositions that was captured under the second image capture conditions,it is arranged to alleviate discontinuity in the image caused by thedifference between the first image capture conditions and the secondimage capture conditions.

Instead of the above it would also be acceptable, in Variant Embodiment#2 of the present invention, to arrange for the image processing unit 33(i.e. the correction unit 33 b) to correct the image data that has beencaptured under the first image capture conditions (i.e. the image datafor the position of attention and the image data, among the image datafor the reference positions, that has been captured under the firstimage capture conditions) on the basis of the second image captureconditions. In this case as well, it is possible to alleviatediscontinuity in the image arising due to the difference between thefirst image capture conditions and the second image capture conditions.

Alternatively, it would also be acceptable to arrange for the imageprocessing unit 33 (i.e. the correction unit 33 b) to correct both theimage data captured under the first image capture conditions and theimage data captured under the second image capture conditions. In otherwords it will be acceptable to arrange to alleviate discontinuity in theimage due to the difference between the first image capture conditionsand the second image capture conditions, by performing correctionprocessing upon the image data for the position for attention that hasbeen captured under the first image capture conditions, upon the imagedata, among the image data for the reference positions, that has beencaptured under the first image conditions, and also upon the image data,among the image data for the reference positions, that has been capturedunder the second image capture conditions.

For example, in Example #1 described above, as correction processing,the image data for the reference pixels that were captured under thefirst image capture conditions (in which the ISO sensitivity was 100)may be multiplied by 400/100, while, as correction processing, the imagedata for the reference pixels that were captured under the second imagecapture conditions (in which the ISO sensitivity was 800) may bemultiplied by 400/800. By doing this, variance in the image data due tothe difference in the image capture conditions is reduced. It should beunderstood that correction processing is performed upon the pixel datafor the pixel for attention by multiplying it by 100/400 after the colorinterpolation processing. By this correction processing, it is possibleto change the pixel data for the pixel for attention after colorinterpolation processing to be the same value as when it was capturedunder the first image capture conditions. Moreover it would also beacceptable, in Example #1 described above, to vary the level ofcorrection processing according to the distance from the boundarybetween the first region and the second region. Thereby, it is possibleto reduce the proportion by which the image data is increased ordecreased by the correction processing, as compared with the case ofExample #1 described above, so that it is possible to reduce the noisegenerated by the correction processing. Although the above explanationhas related to the case of Example #1 described above, the sameprinciples can also be applied to Example #2 above, in a similar manner.

According to this Variant Embodiment #2, in a similar manner to theembodiment described previously, it is possible to perform imageprocessing in an appropriate manner on the image data generated for eachof the regions having different image capture conditions.

Variant Embodiment #3

In the explanation given above, it was arranged, when performingcorrection processing on the image data, to obtain the corrected imagedata by performing calculation on the basis of the difference betweenthe first image capture conditions and the second image captureconditions. Instead of performing such a calculation, it would also beacceptable to obtain the corrected image data by referring to acorrection table. For example, it would be acceptable to read out thecorrected image data by inputting the first image capture conditions andthe second image capture conditions as arguments. Alternatively, itwould also be acceptable to provide a structure in which a correctioncoefficient is read out by inputting the first image capture conditionsand the second image capture conditions as arguments.

Variant Embodiment #4

It would also be acceptable to set an upper limit and a lower limit ofthe image data after correction for the correction processing describedabove. By setting such an upper limit value and lower limit value, it ispossible to impose limitation in order that more correction thannecessary should not be performed. The upper limit value and the lowerlimit value may be determined in advance; or, if a photometric sensor isprovided separately from the image capturing element 32 a, the upperlimit value and the lower limit value may be determined on the basis ofthe output signal from that photometric sensor.

Variant Embodiment #5

In the embodiments described above, examples have been described inwhich photographic subject elements are detected by the control unit 34(i.e. by the setting unit 34 b) on the basis of the live view image, andthe screen of the live view image is subdivided into regions thatinclude those photographic subject elements. However, in VariantEmbodiment #5, it would also be acceptable, if a photometric sensor isprovided separately from the imaging element 32 a, to subdivide theregions on the basis of the output signal from that photometric sensor.

A foreground and a background are subdivided on the basis of the outputsignal from the photometric sensor. In concrete terms, the live viewimage acquired by the imaging element 32 b is subdivided into aforeground region corresponding to a region that is determined to be theforeground from the output signal of the photometric sensor, and abackground region corresponding to a region that is determined to be thebackground from the output signal of the photometric sensor.

Furthermore, the control unit 34 arranges a first region and a secondregion as shown by way of example in FIGS. 16A through 16C with respectto the position corresponding to the foreground region of the imagecapturing surface of the imaging element 32 a. On the other hand, thecontrol unit 34 arranges only a first region on the image capturingsurface of the imaging element 32 a with respect to the positioncorresponding to the background region of the image capturing surface ofthe imaging element 32 a. The control unit 34 employs the first imagefor display, and employs the second image for detection.

According to this Variant Embodiment #5, it is possible to performsubdivision of the live view image acquired by the imaging element 32 binto regions by employing the output signal from the photometric sensor.Furthermore, it would be possible to obtain both the first image fordisplay and the second image for detection for the foreground region,and to obtain only the first image for display for the backgroundregion.

Variant Embodiment #6

In Variant Embodiment #6, as one example of correction processing, theimage processing unit 33 (i.e. the generation unit 33 c) performscontrast adjustment processing. In other words, the generation unit 33 calleviates discontinuity in the image caused by variance between thefirst image capture conditions and the second image capture conditionsby making their gradation curves (i.e. their gamma curves) be different.

For example, suppose that the only difference between the first imagecapture conditions and the second image capture conditions is their ISOsensitivities, and that the ISO sensitivity of the first image captureconditions is 100 while the ISO sensitivity of the second image captureconditions is 800. In this case, among the image data for the referencepositions, the generation unit 33 c compresses the values of the imagedata captured under the second image conditions by ⅛, by making thegradation curve or tone curve flatter.

Alternatively it will also be acceptable, for the generation unit 33 cto expand the values of the image data for the position of attention andthe image data captured under the first image conditions among the imagedata for the reference positions by 8 times, by making the gradationcurve steeper.

According to this Variant Embodiment #6, in a similar manner to theother embodiments described above, it is possible to perform imageprocessing in an appropriate manner upon the image data generated foreach of several different regions that have different image captureconditions. For example, it is possible to suppress discontinuity orstrangeness appearing in the image after image processing that might begenerated due to differences in the image capture conditions at theboundaries between the various regions.

Variant Embodiment #7

In Variant Embodiment #7, it is arranged for the image processing unit33 not to lose the contours of a photographic subject element during theimage processing described above (for example, during the noisereduction processing). In general, smoothing filter processing isemployed when performing noise reduction. Although there is a beneficialeffect with regard to reduction of noise when a smoothing filter isemployed, the boundaries between photographic subject elements maybecome blurred.

Accordingly, for example, the image processing unit 33 (i.e. thegeneration unit 33 c) may compensate for blurring at a boundary at thephotographic subject element described above by performing contrastadjustment processing, in addition to the noise reduction processing oralong with the noise reduction processing. In this Variant Embodiment#7, the image processing unit 33 (i.e. the generation unit 33 c) sets acurve 330 shaped in a letter-S shape (i.e. so-called letter-Sconversion) as a grayscale conversion curve (i.e. as the gradationconversion curve). And, by performing contrast adjustment whileemploying this letter-S conversion, the image processing unit 33 (i.e.the generation unit 33 c) extends the gradations of both the bright dataand the dark data, thereby increasing the gradation levels (i.e. tones)of the bright data (and of the dark data) while also compressing theimage data having intermediate tones and reducing its gradation levels.Due to this, the amount of image data whose brightness is ofintermediate level is reduced, while the amount of image data classifiedas either bright or dark is increased, so that, as a result, it ispossible to compensate blurring at the boundaries of the photographicsubject elements.

According to this Variant Embodiment #7, it is possible to compensateblurring at boundaries of the photographic subject elements byemphasizing the brightness of the image.

Variant Embodiment #8

In Variant Embodiment #8, the image processing unit 33 (i.e. thegeneration unit 33 c) changes the white balance adjustment gain so as toalleviate discontinuity in the image engendered by discrepancy betweenthe first image capture conditions and the second image captureconditions.

For example, in a case in which the image capture conditions that wereapplied for image capture at the position for attention (here supposedto be the first image capture conditions) and the image captureconditions that were applied for image capture at reference positions inthe neighborhood of the position for attention (here supposed to be thesecond image capture conditions) are different from one another, theimage processing unit 33 (i.e. the generation unit 33 c) changes thewhite balance adjustment gain so that the white balance of the imagedata, among the image data for the reference positions, that wasacquired under the second image capture conditions becomes closer to thewhite balance of the image data that was acquired under the first imagecapture conditions.

It should be understood that it would also be acceptable to arrange forthe image processing unit 33 (i.e. the generation unit 33 c) to changethe white balance adjustment gain, so that the white balance of theimage data, among the image data for the reference positions, that wasacquired under the first image capture conditions, and also the whitebalance of the image data for the position for attention, become closerto the white balance of the image data that was acquired under thesecond image capture conditions.

According to this Variant Embodiment #8, for image data that has beengenerated for several regions whose image capture conditions aredifferent from one another, it is possible to alleviate discontinuity inthe image engendered by variance between the first image captureconditions and the second image capture conditions, by matching thewhite balance adjustment gains of one region to the white balanceadjustment gain of another region whose image capture conditions aredifferent.

Variant Embodiment #9

It would also be acceptable to provide a plurality of the imageprocessing units 33, and for them to perform the image processing inparallel. For example, image processing may be performed on image datathat has been captured for a region A of the image capturing unit 32,while also performing image processing on image data that has beencaptured for a region B of the image capturing unit 32. The plurality ofimage processing units 33 may perform the same type of image processing,or may perform different types of image processing. In other words, theplurality of image processing units 33 may perform similar imageprocessing on the image data for the region A and on the image data forthe region B by employing the same parameters and so on; or,alternatively, they may perform different types of image processing onthe image data for the region A and on the image data for the region Bby employing different parameters and so on.

If a plurality of image processing units 33 are provided, then it wouldbe acceptable to perform image processing with one of the imageprocessing units upon image data to which the first image captureconditions have been applied, while performing image processing withanother of the image processing units upon image data to which thesecond image capture conditions have been applied. Moreover, the numberof image processing units is not limited to being two as describedabove; for example, it would also be acceptable to provide the samenumber of image processing units as the number of image captureconditions that can be set. In other words, each of the image processingunits would be tasked with performing image processing for one of theregions to which different image capture conditions have been applied.According to this Variant Embodiment #9, it is possible to progress inparallel the image capturing under different image capture conditionsfor each of the regions, and also the image processing upon the imagedata for the images obtained for each of the above described regions.

Variant Embodiment #10

In the embodiments described above, the camera 1 has been described asan example, but it would also be acceptable to provide a similarstructure to a high-functioning portable telephone handset 250 (refer toFIG. 18 ) such as a smartphone that is equipped with a camera function,or to a mobile device such as a tablet terminal or the like.

Variant Embodiment #11

In the embodiments described above, the camera 1 that is built as asingle integral electronic device incorporating the image capturing unit32 and the control unit 34 has been described by way of example.However, instead of that configuration, it would also be acceptable toprovide an image capturing system 1B in which an image capturing unit 32and a control unit 34 are provided separately, and the image capturingunit 32 is controlled from the control unit 34 via a communicationsystem.

In the following, an example will be explained with reference to FIG. 17in which an image capturing device 1001 that is equipped with an imagecapturing unit 32 is controlled from a display device 1002 to which acontrol unit 34 is provided.

FIG. 17 is a block diagram showing an example of the structure of animage capturing system 1B according to this Variant Embodiment #11. InFIG. 17 , the image capturing system 1B comprises an image capturingdevice 1001 and a display device 1002. In addition to the imagecapturing optical system 31 and the image capturing unit 32 of the typesdescribed in connection with the above embodiments, the image capturingdevice 1001 also comprises a first communication unit 1003. Furthermore,in addition to the image processing unit 33, the control unit 34, thedisplay unit 35, the actuation members 36, and the recording unit 37 asdescribed in connection with the above embodiments, the display device1002 also comprises a second communication unit 1004.

The first communication unit 1003 and the second communication unit 1004are capable of performing bidirectional image data communication by, forexample, a per se known wireless communication technique or opticalcommunication technique or the like.

It should be understood that it would also be acceptable to connect theimage capturing device 1001 and the display device 1002 together bycable connection, so that the first communication unit 1003 and thesecond communication unit 1004 can perform bidirectional image datacommunication.

In the image capturing system 1B, the control unit 34 performs controlof the image capturing unit 32 by performing data communication via thesecond communication unit 1004 and the first communication unit 1003.For example, by predetermined control data being transmitted andreceived between the image capturing device 1001 and the display device1002, the display device 1002 divides the screen into a plurality ofregions, sets different image capture conditions for each of the dividedregions, and reads out photoelectric conversion signals that have beenphotoelectrically converted at each region, on the basis of images suchas described above.

Since, according to this Variant Embodiment #11, the live view imagethat is acquired at the image capturing device 1001 and that istransmitted to the display device 1002 is displayed on the display unit35 of the display device 1002, accordingly the user is able to performremote actuation from the display device 1002, which is located at aposition remote from the image capturing device 1001.

The display device 1002 may, for example, be constituted by a highfunction portable telephone handset 250 such as a smartphone. Moreover,the image capturing device 1001 may be constituted by an electronicdevice that is equipped with an imaging element 100 of the laminatedtype described above.

It should be understood that, although an example has been described inwhich the detection unit 34 a, the setting unit 34 b, the imagecapturing control unit 34 c, and the AF calculation unit 34 d areprovided to the control unit 34 of the display device 1002, it wouldalso be acceptable to arrange for some of the detection unit 34 a, thesetting unit 34 b, the image capturing control unit 34 c, and the AFcalculation unit 34 d to be provided to the image capturing device 1001.

Variant Embodiment #12

As shown by way of example in FIG. 18 , supply of the program to amobile device such as the camera 1 described above, to thehigh-functioning portable telephone handset 250, or to a tablet terminalor the like may be implemented by infrared radiation transmission to themobile device from a personal computer 205 on which the program isstored, or by short distance wireless communication.

Supply of the program to the personal computer 205 may be performed byloading a recording medium 204 such as a CD-ROM or the like upon whichthe program is stored into the personal computer 205, or by loading theprogram into the personal computer 205 by transmission via acommunication line 201 such as a network or the like. If the program istransmitted via such a communication line 201, then the program may bestored in a storage device 203 of a server 202 or the like that isconnected to that communication line.

It would also be possible to transmit the program directly to the mobiledevice by transmission via an access point of a wireless LAN (not shownin the figures) that is connected to the communication line 201.Moreover, it would be possible to load a recording medium 204B such as amemory card or the like upon which the program is stored into the mobiledevice. In this manner, the program may be supplied as a computerprogram product in various formats, such as by provision upon arecording medium or via a communication line or the like.

Second Embodiment

A digital camera will now be explained with reference to FIGS. 19through 25 , as one example of an electronic device that is equippedwith an image processing device according to the second embodiment ofthe present invention. In the following explanation, the same referencesymbols as those employed for the first embodiment will be appended tostructural elements that are similar, and the explanation willprincipally focus upon the points of difference. Features that are notparticularly explained are the same as in the first embodiment. The mainfeature by which this second embodiment differs from the firstembodiment is that, instead of the image processing unit 33 that isprovided in the first embodiment, an image capturing unit (or an imagingunit) 32A further comprises an image processing unit 32 c that has asimilar function to that of the image processing unit 33 of the firstembodiment.

FIG. 19 is a block diagram showing an example of the structure of acamera 1C according to this second embodiment. In FIG. 19 , the camera1C comprises the image capturing optical system 31, the aforementionedimage capturing unit 32A, the control unit 34, the display unit 35, theactuation members 36, and the recording unit 37. The image capturingunit 32A further comprises an image processing unit 32 c that has afunction similar to that of the image processing unit 33 of the firstembodiment.

The image processing unit 32 c includes an input unit 321, a correctionunit 322, and a generation unit 323. Image data from the image capturingelement 32 a is inputted to the input unit 321. The correction unit 322performs pre-processing to correct the image data that has been inputtedas described above. The pre-processing performed by the correction unit322 is the same as the pre-processing performed by the correction unit33 b in the first embodiment. The generation unit 323 performs imageprocessing upon the image data that has been inputted as described aboveand upon the image data after pre-processing and generates an image.This image processing performed by the generation unit 323 is the sameas the image processing performed by the generation unit 33 c in thefirst embodiment.

FIG. 20 is a figure schematically showing a correspondence relationshipin this second embodiment between a plurality of blocks and a pluralityof the correction units 322. In FIG. 20 , a single square frame shown asa rectangle on the image capturing chip 111 represents a single block111 a. In a similar manner, a single square frame shown as a rectangleon an image processing chip 114 that will be described hereinafterrepresents a single correction unit 322.

In this second embodiment, one of the correction units 322 is providedto correspond to each of the blocks 111 a. To put this in anothermanner, one correction unit 322 is provided for each of the blocks,which are the minimum unit regions on the image capturing surface forwhich the image capture conditions can be changed. For example, theblock 111 a in FIG. 20 that is shown by shading and the correction unit322 that is shown by shading are in a relationship of mutualcorrespondence. The correction unit 322 in FIG. 20 that is shown byshading performs pre-processing for the image data from the pixels thatare included in the block 111 a that is shown by shading. Thus, each ofthe correction units 322 performs pre-processing upon the image datafrom the pixels that are included in the corresponding block 111 a.

Since, due to this, it is possible for the pre-processing of the imagedata to be performed by the plurality of correction units 322 inparallel, accordingly it is possible to alleviate the processing burdenupon the correction units 322, and it is possible to generate anappropriate image in a short time period from the image data that hasbeen generated for each of the regions for which the image captureconditions are different.

It should be understood that in the following explanation, whendescribing the relationship between a block 111 a and the pixels thatare included in that block 111 a, that block 111 a may be referred to asbeing the block 111 a to which those pixels belong. Moreover, a block111 a may be referred to as a unit section, and a group having aplurality of the blocks 111 a, in other words a collection in which aplurality of unit subdivisions are grouped together, may be referred toas a “compound section”.

FIG. 21 is a sectional view of a laminated type imaging element (orimage sensor) 100A. In addition to a backside illumination type imagecapturing chip 111, a signal processing chip 112, and a memory chip 113,this laminated type imaging element 100A further comprises an imageprocessing chip 114 that performs the pre-processing and the imageprocessing described above. In other words, the image processing unit 32c described above is provided upon the image processing chip 114.

The image capturing chip 111, the signal processing chip 112, the memorychip 113, and the image processing chip 114 are laminated together, andare mutually electrically connected together by electrically conductivebumps 109 that are made from copper or the like.

A plurality of the bumps 109 are disposed at the mutually opposing facesof the memory chip 113 and the image processing chip 114. These bumps109 are mutually positionally aligned, and, by pressing the memory chip113 and the image processing chip 114 together or the like, the mutuallypositionally aligned bumps 109 are joined together and are mutuallyelectrically connected.

Correction Processing

In a similar manner to the case with the first embodiment, in thissecond embodiment it is arranged, after the regions of the imagecapturing screen have been divided by the setting unit 34 b, to bepossible to set (or to change) the image capture conditions for a regionthat has been selected by the user, or for a region that has beendetermined by the control unit 34. When different image captureconditions have been set for various ones of the subdivided regions,then, according to requirements, the control unit 34 causes thecorrection units 322 to perform correction processing as follows.

1. When Performing Image Processing 1-1 When Image Capture Conditionsfor Pixel for Attention P and Image Capture Conditions for Plurality ofReference Pixels Surrounding Pixel for Attention P are the Same

In this case, in the image processing unit 32 c, correction processingis not performed by the correction units 322, and the generation unit323 performs image processing by employing the image data for theplurality of reference pixels upon which no correction has beenperformed.

1-2 when Image Capture Conditions for Pixel for Attention P and ImageCapture Conditions for at Least One Reference Pixel Among Plurality ofReference Pixels Surrounding Pixel for Attention P are Different

It will be supposed that the image capture conditions that are appliedat the pixel for attention P are the first image capture conditions,that the image capture conditions that are applied at a portion of theplurality of reference pixels are the first image capture conditions,and that the image capture conditions that are applied at the remainderof the reference pixels are the second image capture conditions.

In this case, the correction unit 322 corresponding to a block 111 a towhich a reference pixel to which the second image capture conditionshave been applied belongs performs correction processing as shown in thefollowing Examples #1 through #3 upon the image data for the referencepixel to which the second image capture conditions have been applied.And the generation unit 323 performs image processing for calculatingthe image data for the pixel for attention P by referring to the imagedata for the reference pixel to which the first image capture conditionshave been applied, and to the image data for the reference pixel aftercorrection processing.

Example #1

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their ISOsensitivity, with the ISO sensitivity in the first image captureconditions being 100 while the ISO sensitivity in the second captureconditions is 800, then, as correction processing, the correction unit322 corresponding to the block 111 a, to which the reference pixel towhich the second image capture conditions have been applied belongs,multiplies the image data for the reference pixel by 100/800. By doingthis, variance in the image data due to the difference in the imagecapture conditions is reduced.

Example #2

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their shutterspeed, with the shutter speed in the first image capture conditionsbeing 1/1000 sec. while the shutter speed in the second captureconditions is 1/100 sec., then, as correction processing, the correctionunit 322 corresponding to the block 111 a, to which the reference pixelto which the second image capture conditions have been applied belongs,multiplies the image data for the reference pixel by 1/1000/ 1/100=1/10. By doing this, discrepancy in the image data caused by thedifference in the image capture conditions is reduced.

Example #3

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their frame rate(and with the charge accumulation time being the same), with the framerate in the first image capture conditions being 30 fps while the framerate in the second capture conditions is 60 fps, then, for the imagedata for the reference pixel, in other words for the image data to whichthe second image capture conditions (with frame rate 60 fps) have beenapplied, the correction units 322 corresponding to the block 111 a, towhich the reference pixel to which the second image capture conditionshave been applied belongs, performs correction processing by employingthe image data for the frame images whose starting times of acquisitionare close to those of the frame images that were acquired under thefirst image capture conditions (with frame rate 30 fps). Due to this,variance in the image data due to the difference in the image captureconditions is reduced.

It should be understood that it would also be acceptable to perform thecorrection processing by, on the basis of the previous and subsequentframe images that were acquired under the second image captureconditions (with frame rate 60 fps), performing interpolationcalculation upon the image data for the frame images whose startingtimes of acquisition are close to those of the frame images that wereacquired under the first image capture conditions (with frame rate 30fps).

Moreover, it should also be noted that the same applies to the case inwhich the image capture conditions that were applied to the pixel forattention P are taken as being the second image capture conditions,while the image capture conditions that were applied to reference pixelsaround the pixel for attention P are taken as being the first imagecapture conditions. In other words, in this case, the correction unit322 corresponding to the block 111 a to which the reference pixel towhich the first image capture conditions have been applied performscorrection processing as described in the above Examples #1 through #3upon the image data for the reference pixel.

It should be understood that, as described above, even if the imagecapture conditions are slightly or insubstantially different, they arestill considered to be the same image capture conditions.

On the basis of the image data for the reference pixel to which the sameimage capture conditions as the image capture conditions for the pixelfor attention P have been applied and the image data for the referencepixel that has been corrected by the correction unit 322, the generationunit 323 perform image processing such as pixel defect correctionprocessing, color interpolation processing, contour emphasis processing,and noise reduction processing and the like, in a similar manner to thecase for the image processing unit 33 (i.e. the generation unit 33 c) inthe first embodiment of the present invention.

FIG. 22 is a figure in which the processing of image data (hereinaftertermed “first image data”) from pixels that are included in a partialregion (hereinafter termed the “first region 141”) of the imagecapturing surface to which the first image capture conditions have beenapplied and the processing of image data (hereinafter termed “secondimage data”) from pixels that are included in a partial region(hereinafter termed the “second region 142”) of the image capturingsurface to which the second image capture conditions have been appliedare schematically illustrated.

First image data that has been captured under the first image captureconditions is outputted from each of the pixels included in the firstregion 141, and second image data that has been captured under thesecond image capture conditions is outputted from each of the pixelsincluded in the second region 142. The first image data is outputted tothose correction units 322, among the correction units 322 provided tothe processing chip 114, that correspond to the blocks 111 a to whichthe pixels generating the first image data belong. In the followingexplanation, the plurality of correction units 322, that respectivelycorrespond to the plurality of blocks 111 a to which the pixelsgenerating the first image data belong, will be termed a “firstprocessing unit 151”.

In a similar manner, the second image data is outputted to thosecorrection units 322, among the correction units 322 provided to theprocessing chip 114, that correspond to the blocks 111 a to which thepixels generating the second image data belong. In the followingexplanation, the plurality of correction units 322, that respectivelycorrespond to the plurality of blocks 11 a to which the pixelsgenerating the second image data belong, will be termed a “secondprocessing unit 152”.

For example, if the pixel for attention P is included in the firstregion 141, then the second image data from the reference pixels thatare included in the second region 142 is subjected to the correctionprocessing described above by the second processing unit 152, as shownin FIG. 22 . It should be understood that the second processing unit 152receives from the first processing unit 151, for example, information181 related to the first image capture conditions that is required forreducing variance in the image data due to the difference in the imagecapture conditions.

In a similar manner, for example, if the pixel for attention P isincluded in the second region 142, then the first image data from thereference pixels that are included in the first region 141 is subjectedto the correction processing described above by the first processingunit 151. It should be understood that the first processing unit 151receives from the second processing unit 152 information related to thesecond image capture conditions that is required for reducing variancein the image data due to the difference in the image capture conditions.

It should be understood that, if the pixel for attention P and also thereference pixels are included in the first region 141, then the firstprocessing unit 151 does not perform correction upon the first imagedata from those reference pixels. In a similar manner, if the pixel forattention P and also the reference pixels are included in the secondregion 142, then the second processing unit 152 does not performcorrection upon the second image data from those reference pixels.

Alternatively, it would also be acceptable to arrange for both the imagedata that was captured under the first image capture conditions and theimage data that was captured under the second image capture conditionsto be corrected, respectively, by the first processing unit 151 and thesecond processing unit 152. In other words it would be acceptable toarrange to alleviate discontinuity in the image arising from thedifference between the first image capture conditions and the secondimage capture conditions by performing correction processing upon theimage data for the position of attention that was captured under thefirst image capture conditions, upon the image data, among the imagedata for the reference positions, that was captured under the firstimage capture conditions, and also upon the image data, among the imagedata for the reference positions, that was captured under the secondimage capture conditions.

For example, in Example #1 described above, the image data of thereference pixels that were captured under the first image captureconditions (with ISO sensitivity 100) may be multiplied by 400/100 ascorrection processing, while the image data of the reference pixels thatwere captured under the second image capture conditions (with ISOsensitivity 800) is multiplied by 400/800 as correction processing. Bydoing this, discrepancy in the image data due to the difference in theimage capture conditions is reduced. It should be noted that the pixeldata for the pixel for attention is subjected to correction processingby being multiplied by 100/400 after the color interpolation processing.Due to this correction processing, it is possible to change the pixeldata of the pixel for attention after color interpolation processing toa value similar to the value obtained under the first image captureconditions. Moreover, in Example #1 above, it would also be acceptableto change the level of the correction processing according to thedistances of the first region and the second region from a boundary. Andit would also be possible to reduce the rate of increase or decrease ofthe image data in the correction processing as compared with the case ofExample #1 above, thereby enabling reduction of the noise caused by thecorrection processing. Although the above explanation relates to Example#1 described above, it can also be applied, in a similar manner, toExample #2 described above as well.

On the basis of the image data from the first processing unit 151 andfrom the second processing unit 152, the generation unit 323 performsimage processing such as pixel defect correction processing, colorinterpolation processing, contour emphasis processing, noise reductionprocessing, and the like, and outputs the image data after imageprocessing.

It should be understood that, if the pixel for attention P is positionedin the second region 142, then it will be acceptable to arrange for thefirst processing unit 151 to perform correction processing upon thefirst image data from all of the pixels that are included in the firstregion 141; or, alternatively, it will also be acceptable to arrange forthe first processing unit 151 to perform correction processing upon onlythe first image data from those pixels, among the pixels that areincluded in the first region 141, that are likely to be employed forinterpolation of the pixel for attention P in the second region 142. Ina similar manner, if the pixel for attention P is positioned in thefirst region 141, then it will be acceptable to arrange for the secondprocessing unit 152 to perform correction processing upon the secondimage data from all of the pixels that are included in the second region142; or, alternatively, it will also be acceptable to arrange for thesecond processing unit 152 to perform correction processing upon onlythe second image data from those pixels, among the pixels that areincluded in the second region 142, that are likely to be employed forinterpolation of the pixel for attention P in the first region 141.

2. When Performing Focus Detection Processing

In a similar manner to the case with the first embodiment, the controlunit 34 (i.e. the AF calculation unit 34 d) performs focus detectionprocessing by employing signal data (i.e. image data) corresponding to apredetermined position (i.e. the focus detection position) in the imagecapturing screen. It should be understood that, if different imagecapture conditions are set for the various divided regions, and thefocus detection position for AF operation is positioned at a boundaryportion between the divided regions, in other words if the focusdetection position is divided into two between a first region and asecond region, then, in this second embodiment, as explained in 2-2below, the control unit 34 (i.e. the AF calculation unit 34 d) causesthe correction unit 322 so as to perform correction processing on thesignal data for focus detection from at least one region.

2-1. If Signal Data to which the First Image Capture Conditions havebeen Applied and Signal Data to which the Second Image CaptureConditions have been Applied are not Mixed Together in the Signal Datafrom the Pixels in the Frame 170 of FIG. 13

In this case the correction unit 322 does not perform any correctionprocessing, and the control unit 34 (i.e. the AF calculation unit 34 d)performs the focus detection processing by employing the signal datafrom the pixels for focus detection shown by the frame 170 just as it iswithout alteration.

2-2. If Signal Data to which the First Image Capture Conditions havebeen Applied and Signal Data to which the Second Image CaptureConditions have been Applied are Mixed Together in the Signal Data fromthe Pixels in the Frame 170 of FIG. 13

In this case, the control unit 34 (i.e. the AF calculation unit 34 d)causes the correction unit 322 corresponding to the block 111 a to whichthe pixel, among the pixels within the frame 170, to which the secondimage capture conditions have been applied belongs, to performcorrection processing as shown in the following Examples #1 through #3.And the control unit 34 (i.e. the AF calculation unit 34 d) performs thefocus detection processing by employing the signal data for the pixelsto which the first image capture conditions have been applied, and alsothe signal data after the correction processing.

Example #1

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their ISOsensitivity, with the ISO sensitivity in the first image captureconditions being 100 while the ISO sensitivity in the second captureconditions is 800, then, as correction processing, the correction unit322 corresponding to the block 111 a, to which the pixels to which thesecond image capture conditions have been applied belong, multiplies thesignal data captured under the second image capture conditions by100/800. By doing this, variance between the signal data caused by thedifferent image capture conditions are reduced.

Example #2

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their shutterspeed, with the shutter speed in the first image capture conditionsbeing 1/1000 sec. while the shutter speed in the second captureconditions is 1/100 sec., then, as correction processing, the correctionunit 322 corresponding to the block 111 a, to which the pixels to whichthe second image capture conditions have been applied belong, multiplythe signal data captured under the second image capture conditions by1/1000/ 1/100= 1/10. By doing this, variance between the signal datacaused by the difference in the image capture conditions are reduced.

Example #3

If, for example, the only difference between the first image captureconditions and the second image capture conditions is their frame rate(with the charge accumulation time being the same), with the frame ratein the first image capture conditions being 30 fps while the frame ratein the second capture conditions is 60 fps, then, as correctionprocessing, the correction unit 322 corresponding to the block 111 a, towhich the pixels to which the second image capture conditions have beenapplied belong, performs correction processing upon the signal datacaptured under the second image capture conditions (i.e. at 60 fps) byemploying the signal data for the frame images whose starting times ofacquisition are close to the frame images that were acquired under thefirst image capture conditions (with frame rate 30 fps). By doing this,variance in the signal data due to the difference in the image captureconditions is reduced.

It should be understood that it would also be acceptable to perform, asthe correction processing, on the basis of the previous and subsequentframe images that were acquired under the second image captureconditions (with frame rate 60 fps), interpolation calculation of thesignal data for the frame image whose starting time of acquisition isclose to that of the frame image acquired under the first image captureconditions (with frame rate 30 fps).

It should be understood that, as described above, even if the imagecapture conditions are slightly or insubstantially different, they arestill considered to be the same image capture conditions.

Moreover, an example has been described above in which the correctionprocessing is performed upon the signal data, among the signal data,that has been captured under the second image capture conditions; but itwould also be acceptable to perform the correction processing upon thesignal data, among the signal data, that has been captured under thefirst image capture conditions.

Yet further it would also be acceptable to arrange for the correctionprocessing to be performed upon, among the signal data, both the signaldata that has been captured under the first image capture conditions andthe signal data that has been captured under the second image captureconditions, whereby the difference between the signal data aftercorrection processing could be made yet smaller.

FIG. 23 is a figure in which the processing of the first image data andthe processing of the second image data relating to focus detectionprocessing are schematically shown.

First signal data that has been captured under the first image captureconditions is outputted from each pixel that is included in the firstregion 141, and second signal data that has been captured under thesecond image capture conditions is outputted from each pixel that isincluded in the second region 142. The first signal data from the firstregion 141 is outputted to the first processing unit 151. In a similarmanner, the second signal data from the second region 142 is outputtedto the second processing unit 152.

When performing correction processing upon, among the signal data, thesignal data that has been captured under the second image captureconditions so as to reduce the difference between the signal data aftercorrection processing and the signal data that was captured under thefirst image capture conditions, the second processing unit 152 performsthat processing. The second processing unit 152 performs the correctionprocessing described above upon the second signal data from the pixelsincluded in the second region 142. It should be understood that thesecond processing unit 152 receives information 181 relating to thefirst image capture conditions that is required for reducing thedifference in the signal data caused by the different image captureconditions, for example from the first processing unit 151.

On the other hand it should be understood that, in this case in whichthe difference between the signal data after correction processing andthe signal data that was captured under the first image captureconditions is reduced by performing correction processing upon, amongthe signal data, the signal data that was captured under the secondimage capture conditions, the first processing unit 151 does not performcorrection of the first signal data.

Furthermore, when reducing the difference between the signal data aftercorrection processing and the signal data captured under the firstcapture conditions by performing correction processing upon the signaldata, among the signal data, that was captured under the first imagecapture conditions, then the first processing unit 151 performsprocessing. The first processing unit 151 performs the correctionprocessing described above upon the first signal data from the pixelsthat are included in the first region 141. It should be understood thatthe first processing unit 151 receives from the second processing unit152 information relating to the second image capture conditions,required in order to reduce variance in the signal data due to thedifferent image capture conditions.

It should be noted that, in this case in which the difference betweenthe signal data after correction processing and the signal data that wascaptured under the first image capture conditions is reduced byperforming correction processing upon, among the signal data, the signaldata that has been captured under the first image capture conditions,then the second processing unit 152 does not correct the second signaldata.

Even further, for reducing the difference between the two sets of signaldata after the correction processing by performing correction processingupon, among the signal data, both the signal data captured under thefirst image capture conditions and the signal data captured under thesecond image capture conditions, then both the first processing unit 151and the second processing unit 152 perform processing. The firstprocessing unit 151 performs the correction processing described aboveupon the first signal data from the pixels that are included in thefirst region 141, and the second processing unit 152 performs thecorrection processing described above upon the second signal data fromthe pixels that are included in the second region 142.

The AF calculation unit 34 d performs the focus detection processing onthe basis of the signal data from the first processing unit 151 and fromthe second processing unit 152, and outputs a drive signal for causingthe focusing lens of the image capturing optical system 31 to shift toits focusing position on the basis of the results of this calculation.

3. When Performing Photographic Subject Detection Processing

When different image capture conditions are set for the varioussubdivided regions and the search range 190 includes a boundary betweenthe subdivided regions, then, in the present embodiment, as explained in3-2 below, the control unit 34 (i.e. the object detection unit 34 a)causes the correction unit 322 to perform correction processing upon theimage data for at least one region within the search range 190.

3-1. If Image Data to which the First Image Capture Conditions have beenApplied and Image Data to which the Second Image Capture Conditions havebeen Applied are not Mixed Together in the Image Data of the SearchRange 190 of FIG. 14

In this case, the correction unit 322 does not perform any correctionprocessing, and the control unit 34 (i.e. the object detection unit 34a) performs photographic subject detection processing by employing theimage data that constitutes the search range 190 just as it is withoutalteration.

3-2. If Image Data to which the First Image Capture Conditions have beenApplied and Image Data to which the Second Image Capture Conditions havebeen Applied are Mixed Together in the Image Data of the Search Range190 of FIG. 14

In this case, the control unit 34 (i.e. the object detection unit 34 a)causes the correction unit 322 corresponding to the block 111 a to whichthe pixels, in the image of the search range 190, to which the secondimage capture conditions have been applied belong, to perform correctionprocessing as explained in Examples #1 through #3 described above forperforming focus detection processing. And the control unit 34 (i.e. theobject detection unit 34 a) performs the photographic subject detectionprocessing by employing the image data for the pixels to which the firstimage capture conditions have been applied, and also the image dataafter the correction processing described above.

FIG. 24 is a figure in which the processing of the first image data andthe processing of the second image data relating to the photographicsubject detection processing are schematically shown. The correctionprocessing performed by the first processing unit 151 and/or the secondprocessing unit 152 is the same as the correction processing of FIG. 23for the case of performing the focus detection processing.

The object detection unit 34 a performs processing for detection of thephotographic subject elements on the basis of the image data from thefirst processing unit 151 and from the second processing unit 152, andoutputs the results of this detection.

4. When Setting Image Capture Conditions

The case will now be explained in which, in the state that the imagecapturing screen has been subdivided into regions and different imagecapture conditions have been set for the different subdivided regions,photometry is newly performed and the exposure conditions are newlydetermined.

4-1. If Image Data to which the First Image Capture Conditions have beenApplied and Image Data to which the Second Image Capture Conditions havebeen Applied are not Mixed Together in the Image Data of the PhotometricRange

In this case, the correction unit 322 does not perform any correctionprocessing, and the control unit 34 (i.e. the setting unit 34 b)performs the exposure calculation processing by employing the image datathat constitutes the photometric range just as it is without alteration.

4-2. If Image Data to which the First Image Capture Conditions have beenApplied and Image Data to which the Second Image Capture Conditions havebeen Applied are Mixed Together in the Image Data of the PhotometricRange

In this case, the control unit 34 (i.e. the setting unit 34 b) causesthe correction unit 322 corresponding to the block 111 a to which thepixels, in the image data of the photometric range, to which the secondimage capture conditions have been applied belong, to perform correctionprocessing as shown in Examples #1 through #3 described above forperforming focus detection processing. And the control unit 34 (i.e. thesetting unit 34 b) performs the exposure calculation processing byemploying the image data after the correction processing describedabove.

FIG. 25 is a figure in which the processing of the first image data andthe processing of the second image data are schematically shown,relating to setting of image capture conditions such as for exposurecalculation processing and so on. The correction processing performed bythe first processing unit 151 and/or the second processing unit 152 isthe same as the correction processing of FIG. 23 described above for thecase of performing the focus detection processing.

On the basis of the image data from the first processing unit 151 andfrom the second processing unit 152, the setting unit 34 b performscalculation processing for image capture conditions such as for exposurecalculation processing and so on, subdivides the image capturing screenof the image capturing unit 32 into a plurality of regions containingthe photographic subject elements that have been detected, and thenresets the image capture conditions for each of this plurality ofregions on the basis of the results of the above calculation.

According to this second embodiment as explained above, the followingadvantageous operational effects are obtained.

(1) The camera 1C comprises the image capturing element 32 a that iscapable of capturing images under image capture conditions changed foreach unit section of the image capturing surface, and that generates thefirst image data from the first region having at least one unit sectionin which image capture is performed under the first image captureconditions, and the second image data from the second region having atleast one unit section in which image capture is performed under thesecond image capture conditions that are different from the first imagecapture conditions. And the camera 1C comprises a plurality of thecorrection units 322 that are provided to correspond to each of the unitsections, or to each of compound sections each having a plurality ofunit sections, and that are capable of correcting the image data fromthe respective unit sections, or from the unit section within therespective compound sections. The correction units 322 that correspondto the unit sections or to the compound sections within the secondregion correct the first image data by the second image captureconditions. The image capturing element 32 a is provided at the backsideillumination type image capturing chip 111. And the plurality ofcorrection units 322 are provided at the image processing chip 114.

As a result it is possible to alleviate the processing burden upon thecorrection units 322, since it is possible to perform the correctionprocessing of the image data with a plurality of the correction units322 in parallel.

(2) The backside illumination type image capturing chip 111 and theimage processing chip 114 are laminated together or stacked one on topof another. Due to this, it is possible to connect the image capturingelement 32 a and the image processing unit 32 c together simply andeasily.

(3) The camera 1C comprises the generation unit 323 that generates animage on the basis of the first image data corrected by the correctionunits 322, and of the second image data. Since, due to this, thepre-processing by the plurality of correction units 322 is performed ina short time period by parallel processing, accordingly it is possibleto shorten the time period until the image is generated.

(4) The camera 1C comprises the image capturing element 32 a that iscapable of capturing an image under image capture conditions changed foreach unit sections of the image capturing surface, and that generatesthe first image data from the first region having at least one unitsection in which an optical image incident via the image capturingoptical system is captured under the first image capture conditions, andthe second image data from the second region having at least one unitsection in which the incident optical image is captured under the secondimage capture conditions that are different from the first image captureconditions. And the camera 1C comprises a plurality of the correctionunits 322 that are provided to correspond to each of the unit sections,or to each of compound sections each having a plurality of unit section,and that are capable of correcting the image data from the respectiveunit sections, or from the unit section within the respective compoundsections. The camera 1C also comprises the AF calculation unit 34 d thatdetects information for shifting the image capturing optical system. Thecorrection units 322 that correspond to the unit sections or to thecompound sections within the first region correct the first image dataso as to reduce a difference from the second image data. The AFcalculation unit 34 d detects information for shifting the imagecapturing optical system on the basis of the first image data that hasbeen corrected by the correction units 322, and of the second imagedata. The image capturing element 32 a is provided at the backsideillumination type image capturing chip 111. And the plurality ofcorrection units 322 are provided at the image processing chip 114.

Since it is possible to perform the correction processing of the imagedata with the plurality of correction units 322 that perform processingin parallel, accordingly, along with it being possible to alleviate theprocessing burden upon the correction units 322, also it is possible forthe pre-processing to be performed by the plurality of correction units322 in a short time period by parallel processing, and thus it ispossible to shorten the time period until focus detection processing bythe AF calculation unit 34 d starts, which makes a contribution tospeeding up the focus detection processing.

(5) The camera 1C comprises the image capturing element 32 a that iscapable of capturing an image under image capture conditions changed foreach unit sections of the image capturing surface, and that generatesthe first image data from the first region having at least one unitsections in which an incident image of the photographic subject iscaptured under the first image capture conditions, and the second imagedata from the second region having at least one unit section in which anincident image of the photographic subject is captured under the secondimage capture conditions that are different from the first image captureconditions. And the camera 1C comprises the plurality of correctionunits 322 that are provided to correspond to each of the unit sections,or to each of compound sections each having a plurality of the unitsubdivisions, and that are capable of correcting the image data from thecorresponding unit sections, or from the unit sections within thecorresponding compound section. The camera 1C also comprises the objectdetection unit 34 a that detects the objects from the photographicsubject image. The correction units 322 that correspond to the unitsections or to the compound sections within the first region correct thefirst image data so as to reduce the difference of its values from thevalues of the second image data. The object detection unit 34 a detectsthe objects from the photographic subject image, on the basis of thefirst image data that has been corrected by the correction units 322,and the second image data. The image capturing element 32 a is providedat the backside illumination type image capturing chip 111. And theplurality of correction units 322 are provided at the image processingchip 114.

Since it is possible to perform the correction processing of the imagedata with the plurality of correction units 322 that operate by parallelprocessing, accordingly, along with it being possible to alleviate theprocessing burden upon the correction units 322, also it is possible forthe pre-processing to be performed by the plurality of correction units322 in a short time period by parallel processing, and thus it ispossible to shorten the time period until photographic subject detectionprocessing by the object detection unit 34 a starts, which makes acontribution to speeding up the photographic subject detectionprocessing.

(6) The camera 1C comprises the image capturing element 32 a that iscapable of capturing an image with different image capture conditionsfor each unit sections of the image capturing surface, and thatgenerates the first image data from the first region having at least oneunit section in which an incident optical image is captured under thefirst image capture conditions, and the second image data from thesecond region having at least one unit section in which an incidentoptical image is captured under the second image capture conditions thatare different from the first image capture conditions. And the camera 1Ccomprises a plurality of the correction units 322 that are provided tocorrespond to each of the unit sections, or to each of compoundsubdivisions each having a plurality of unit sections, and that arecapable of correcting the image data from the corresponding unitsections, or from the unit sections within the corresponding compoundsections. The camera 1C also comprises the setting unit 34 b that setsthe image capture conditions. The correction units 322 that correspondto the unit sections or to the compound sections within the first regioncorrect the first image data so as to reduce a difference from thesecond image data. The setting unit 34 b sets the image captureconditions on the basis of the first image data that has been correctedby the correction units 322, and of the second image data. The imagecapturing element 32 a is provided at the backside illumination typeimage capturing chip 111. And the plurality of correction units 322 areprovided at the image processing chip 114.

Since it is possible to perform the correction processing of the imagedata with the plurality of correction units 322 that operate by parallelprocessing, accordingly, along with it being possible to alleviate theprocessing burden upon the correction units 322, also it is possible forthe pre-processing to be performed by the plurality of correction units322 in a short time period by parallel processing, and thus it ispossible to shorten the time period until setting processing for theimage capture conditions by the setting unit 34 b starts, which makes acontribution to speeding up the processing for setting the image captureconditions.

———Variants of Second Embodiment———

The following modifications also come within the scope of the presentinvention, and moreover it would be possible to combine one or aplurality of these modifications with one another, and/or with theembodiments described above.

Variant Embodiment #13

The processing of the first image data and the second image data whenthe first region and the second region are arranged in the imagecapturing surface of the image capturing element 32 a as shown in FIGS.16A through 16C for Variant Embodiment #1 of the first embodiment of thepresent invention will now be explained.

In this Variant Embodiment as well, in a similar manner to VariantEmbodiment #1, in any of the cases shown in FIGS. 16A through 16C, afirst image based upon the image signals read out from the first regionand a second image based upon the image signals read out from the secondregion are generated according to the pixel signals read out from theimage capturing element 32 a that has performed capture of one frame. Inthis Variant Embodiment as well, in a similar manner to VariantEmbodiment #1, the control unit 34 employs the first image for displayand employs the second image for detection.

The image capture conditions set for the first region for capture of thefirst image will be termed the “first image capture conditions”, and theimage capture conditions set for the second region for capture of thesecond image will be termed the “second image capture conditions”. Thecontrol unit 34 may make the first image capture conditions and thesecond image capture conditions be different from one another.

1. With reference to FIG. 26 , an example will now be explained for thecase in which the first image capture conditions set for the firstregion are the same over the entire first region of the image capturingscreen, and also the second image capture conditions set for the secondregion are the same over the entire second region of the imaging screen.FIG. 26 is a figure schematically showing the processing of the firstimage data and the second image data.

The first image data captured under the first image capture conditionsis outputted from each pixel included in the first region 141, and thesecond image data captured under the second image capture conditions isoutputted from each pixel included in the second region 142. The firstimage data from the first region 141 is outputted to the firstprocessing unit 151. In a similar manner, the second image data from thesecond region 142 is outputted to the second processing unit 152.

In this example, the first processing unit 151 does not perform anycorrection upon the first image data from the reference pixels belongingto the first region, since the first image capture conditions are thesame over the entire first region of the image capturing screen.Furthermore, the second processing unit 152 does not perform anycorrection of the second image data to be employed in focus detectionprocessing, photographic subject detection processing, or exposurecalculation processing, since the second image capture conditions arethe same over the entire second region of the image capturing screen.However, the second processing unit 152 does perform correctionprocessing upon the second image data that is employed for interpolationof the first image data, in order to reduce a difference in the imagedata caused by the difference in the first image capture conditions andthe second image capture conditions. The second processing unit 152outputs the second image data after correction processing to the firstprocessing unit 151, as shown by an arrow sign 182. It should beunderstood that it would also be acceptable for the second processingunit 152 to output the second image data after correction processing tothe generation unit 323, as shown by a broken arrow sign 183.

The second processing unit 152 receives from the first processing unit151, for example, information 181 relating to the first image captureconditions required for reducing variance in the image data due to thedifference in the image capture conditions.

On the basis of the first image data from the first processing unit 151and the second image data upon which correction processing has beenperformed by the second processing unit 152, the generation unit 323performs image processing such as pixel defect correction processing,color interpolation processing, contour emphasis processing, noisereduction processing, and so on, and outputs the image data after thisimage processing.

The object detection unit 34 a performs processing to detect thephotographic subject elements on the basis of the second image data fromthe second processing unit 152, and outputs the results of thisdetection.

On the basis of the second image data from the second processing unit152, the setting unit 34 b performs calculation processing of imagecapture conditions such as for exposure calculation processing and soon, divides the image capturing screen of the image capturing unit 32into a plurality of regions that include the photographic subjectelements that have been detected, and then resets the image captureconditions for this plurality of regions on the basis of the results ofthe above calculation.

And the AF calculation unit 34 d performs focus detection processing onthe basis of the second signal data from the second processing unit 152,and outputs a drive signal for shifting the focusing lens of the imagecapturing optical system 31 to a focusing position on the basis of theresults of this calculation.

2. With reference to FIG. 27 , another example will now be explained forthe case in which the first image capture conditions set for the firstregion are different for different areas of the image capturing screen,while the second image capture conditions set for the second region arethe same over the entire second region of the image capturing screen.FIG. 27 is a figure schematically showing the processing of the firstimage data and the second image data.

The first image data captured under the first image capture conditions,which are different according to the areas of the image capturingscreen, is outputted from each pixel included in the first region 141,and the second image data captured under the second image captureconditions, which are the same over the entire second region of theimage capturing screen, is outputted from each pixel included in thesecond region 142. The first image data from the first region 141 isoutputted to the first processing unit 151. In a similar manner, thesecond image data from the second region 142 is outputted to the secondprocessing unit 152.

As described above, in this example, the first image capture conditionsthat are set for the first region vary, depending upon the areas on theimage capturing screen. In other words, depending upon the partialregion within the first region, the first image capture conditions aredifferent. If different first image capture conditions are set for apixel for attention P and for a reference pixel even though they areboth positioned within the first region, then the first processing unit151 performs similar correction processing to the correction processingdescribed in 1-2 above upon the first image data from that referencepixel. It should be understood that, if the first image captureconditions that are set for the pixel for attention P and for thereference pixel are the same, then the first processing unit 151 doesnot perform any correction processing upon the first image data fromthat reference pixel.

In this example, since the second image capture conditions that are setfor the second region of the image capturing screen are the same overthe entire extent of the second region, accordingly the secondprocessing unit 152 does not perform correction upon the second imagedata to be employed in focus detection processing, subject detectionprocessing, and exposure calculation processing. However, for the secondimage data that is employed for interpolation of the first image data,the second processing unit 152 does perform correction processing inorder to reduce variance in the image data due to the difference in theimage capture conditions for the pixel for attention P that is includedin the first region and the second image capture conditions. It shouldbe understood that the second processing unit 152 outputs the secondimage data after correction processing to the first processing unit 151.Moreover, it should be understood that it would also be acceptable forthe second processing unit 152 to output the second image data aftercorrection processing to the generation unit 323.

The second processing unit 152 receives from the first processing unit151, for example, information 181 relating to the image captureconditions for the pixel for attention P that is included in the firstregion, required for reducing variance in the image data caused by thedifferent image capture conditions.

On the basis of the first image data from the first processing unit 151and the second image data that has been subjected to correctionprocessing by the second processing unit 152, the generation unit 323performs image processing such as pixel defect correction processing,color interpolation processing, contour emphasis processing, noisereduction processing, and so on, and outputs the image data after thisimage processing.

The object detection unit 34 a performs processing to detect thephotographic subject elements on the basis of the second image data fromthe second processing unit 152, and outputs the results of thisdetection.

On the basis of the second image data from the second processing unit152, the setting unit 34 b performs calculation processing of the imagecapture conditions such as for exposure calculation processing and soon, divides the image capturing screen of the image capturing unit 32into a plurality of regions that include the photographic subjectelements that have been detected, and then resets the image captureconditions for this plurality of regions on the basis of the results ofthe above calculation.

And the AF calculation unit 34 d performs focus detection processing onthe basis of the second signal data from the second processing unit 152,and outputs a drive signal for shifting the focusing lens of the imagecapturing optical system 31 to a focusing position on the basis of theresults of this calculation.

3. With reference to FIG. 28 , yet another example will now be explainedfor the case in which the first image capture conditions that are setfor the first region are the same over the entire first region of theimage capturing screen, while the second image capture conditions setfor the second region are different for different areas of the imagecapturing screen. FIG. 28 is a figure schematically showing theprocessing of the first image data and the second image data.

The first image data captured under the first image capture conditions,which are the same over the entire first region of the image capturingscreen, is outputted from each pixel included in the first region 141,and the second image data captured under the second image captureconditions, which are different depending upon the area of the imagecapturing screen, is outputted from each pixel included in the secondregion 142. The first image data from the first region 141 is outputtedto the first processing unit 151. In a similar manner, the second imagedata from the second region 142 is outputted to the second processingunit 152.

Since, in this example, the first image capture conditions that are setfor the first region of the image capturing screen are the same over theentire first region, accordingly the first processing unit 151 does notperform any correction upon the first image data from the referencepixels that are included in the first region.

Furthermore, in this example, since the second image capture conditionsthat are set for the second region are different for different areas ofthe image capturing screen, accordingly the second processing unit 152performs the following correction processing upon the second image data.For example, the second processing unit 152 performs correctionprocessing upon second image data, among the second image data, that hasbeen captured under certain image conditions so as to reduce thedifference between the second image data after the correction processingand the second image data that has been captured under other imagecapture conditions that are different from the abovementioned certainimage conditions.

In this example, the second processing unit 152 performs correctionprocessing upon the second image data that is employed for interpolationof the first image data, so as to reduce variance in the image datacaused by the difference between the image capture conditions for thepixel for attention P that is included in the first region and thesecond image capture conditions. The second processing unit 152 outputsthe second image data after correction processing to the firstprocessing unit 151. It should be understood that it would also beacceptable for the second processing unit 152 to output the second imagedata after correction processing to the generation unit 323. Moreover itshould be noted that, in FIG. 26 , the output of the second image dataafter correction processing to the first processing unit 151 is denotedby the reference symbol 182, while the output of the second image dataafter correction processing to the generation unit 323 is denoted by thereference symbol 183.

The second processing unit 152 receives from the first processing unit151, for example, information 181 relating to the image captureconditions for the pixel for attention P that is included in the firstregion, required for reducing variance in the image data due to thedifferent image capture conditions.

On the basis of the first image data from the first processing unit 151and the second image data that has been subjected to correctionprocessing by the second processing unit 152, the generation unit 323performs image processing such as pixel defect correction processing,color interpolation processing, contour emphasis processing, noisereduction processing, and so on, and outputs the image data after imageprocessing.

The object detection unit 34 a performs processing to detect theelements of the photographic subject on the basis of the second imagedata that has been captured under the certain image capture conditionsand that has been subjected to correction processing by the secondprocessing unit 152, and the second image data that has been capturedunder other image capture conditions, and outputs the results of thisdetection.

On the basis of the second image data that has been captured under thecertain image capture conditions and that has been subjected tocorrection processing by the second processing unit 152, and the secondimage data that has been captured under the other processing conditions,the setting unit 34 b performs calculation processing for image captureconditions such as exposure calculation processing and so on. Thesetting unit 34 b divides the image capturing screen of the imagecapturing unit 32 into a plurality of regions that include thephotographic subject elements that have been detected, and then resetsthe image capture conditions for this plurality of regions on the basisof the results of the above calculation.

And the AF calculation unit 34 d performs focus detection processing onthe basis of the second signal data that has been captured under thecertain image capture conditions and that has been subjected tocorrection processing by the second processing unit 152, and the secondsignal data that has been captured under the other image captureconditions; and then the AF calculation unit 34 d outputs a drive signalfor shifting the focusing lens of the image capturing optical system 31to its focusing position on the basis of the results of thiscalculation.

4. With reference to FIG. 29 , still yet another example will now beexplained for the case in which the first image capture conditions setfor the first region are different for different areas of the imagecapturing screen, and also the second image capture conditions set forthe second region are different for different areas of the imagecapturing screen. FIG. 29 is a figure schematically showing theprocessing of the first image data and the second image data.

The first image data captured under the first image capture conditions,which vary according to the areas of the image capturing screen, isoutputted from each pixel included in the first region 141, and thesecond image data captured under the second image capture conditions,which also vary according to the areas of the image capturing screen, isoutputted from each pixel included in the second region 142. The firstimage data from the first region 141 is outputted to the firstprocessing unit 151. In a similar manner, the second image data from thesecond region 142 is outputted to the second processing unit 152.

As described above, in this example, the first image capture conditionsthat are set for the first region of the image capturing screen differaccording to the areas of the image capturing screen. In other words,the first imaging conditions are different for different partial regionswithin the first region. If the first image conditions that are set forthe pixel for attention P and for a reference pixel both of which arepositioned within the first region are different, then the firstprocessing unit 151 performs correction processing upon the first imagedata from that reference pixel, similar to the correction processingdescribed in 1-2 above. Furthermore, it should be understood that, ifthe first image capture conditions that are set for the pixel forattention P and for a reference pixel are the same, then the firstprocessing unit 151 does not perform any correction processing upon thefirst image data from that reference pixel.

Furthermore, in this example, since the second image capture conditionsthat are set for the second region differ according to the areas of theimage capturing screen, accordingly the second processing unit 152performs correction processing as in the case 3. described above.

On the basis of the first image data from the first processing unit 151and the second image data that has been subjected to correctionprocessing by the second processing unit 152, the generation unit 323performs image processing such as pixel defect correction processing,color interpolation processing, contour emphasis processing, noisereduction processing, and so on, and then outputs the image data afterimage processing.

On the basis of the second image data that has been captured undercertain image capture conditions and the second image data that has beencaptured under other image capture conditions, that both have beensubjected to correction processing by the second processing unit 152,the object detection unit 34 a performs processing for detection of thephotographic subject elements, and outputs the result of this detection.

On the basis of the second image data that has been captured under thecertain image capture conditions and the second image data that has beencaptured under the other image capture conditions, that both have beensubjected to correction processing by the second processing unit 152,the setting unit 34 b performs calculation processing for the imagecapture conditions such as exposure calculation processing and so on.And, the setting unit 34 b divides the image capturing screen of theimage capturing unit 32 into a plurality of regions that include thephotographic subject elements that have been detected, and then resetsthe image capture conditions for this plurality of regions on the basisof the results of the above calculation.

And the AF calculation unit 34 d performs focus detection processing onthe basis of the second signal data that has been captured under thecertain image capture conditions and the second signal data that hasbeen captured under the other image conditions, that both have beensubjected to correction processing by the second processing units 152;and then the AF calculation unit 34 d outputs a drive signal forshifting the focusing lens of the image capturing optical system 31 toits focusing position on the basis of the results of this calculation.

Variant Embodiment #14

In the second embodiment described above, one of the correction units322 corresponds to one of the blocks 111 a (i.e. to a unit section).However, it would also be acceptable to arrange for one of thecorrection units 322 to correspond to a compound block (i.e. to acompound section) that incorporates a plurality of the blocks 111 a(i.e. a plurality of unit sections). In this case, the correction unit322 sequentially corrects the image data from the pixels belonging tothe plurality of blocks 111 a included in the compound section. Evenwhen a plurality of the correction units 322 are provided torespectively correspond to the compound blocks each having a pluralityof the blocks 111 a, since it is possible to perform the correctionprocessing of the image data with this plurality of correction units 322in parallel, accordingly it is possible to alleviate the burden ofprocessing upon the correction units 322, and it is possible togenerate, in a short time period, an appropriate image from the imagedata generated for the various regions for which the image captureconditions are different.

Variant Embodiment #15

In the second embodiment described above, the generation unit 323 wasprovided internally to the image capturing unit 32A. However, it wouldalso be possible to provide the generation unit 323 externally to theimage capturing unit 32A. Even when the generation unit 323 is providedexternally to the image capturing unit 32A, it is still possible toachieve similar advantageous operational effects to the advantageousoperational effects described above.

Variant Embodiment #16

In the second embodiment described above, in addition to the backsideillumination type image capturing chip 111, the signal processing chip112, and the memory chip 113, the laminated type imaging element 100Awas further provided with the image processing chip 114 that performedthe pre-processing and the image processing described above. However, itwould also be acceptable not to provide the image processing chip 114 tothe laminated type imaging element 100A, but rather to provide the imageprocessing unit 32 c to the signal processing chip 112.

Variant Embodiment #17

In the second embodiment described above, the second processing unit 152receives from the first processing unit 151 information relating to thefirst capture conditions, required for reducing variance in the imagedata due to the difference in the image capture conditions.Additionally, the first processing unit 151 receives from the secondprocessing unit 152 information relating to the second captureconditions, required for reducing variance in the image data due to thedifference in the image capture conditions. However, it would also beacceptable for the second processing unit 152 to receive from the driveunit 32 b or from the control unit 34 information relating to the firstimage capture conditions, required for reducing variance in the imagedata caused by in the different image capture conditions. In a similarmanner, it would also be acceptable for the first processing unit 151 toreceive from the drive unit 32 b or from the control unit 34 informationrelating to the second image capture conditions, required for reducingvariance in the image data due to the difference in the image captureconditions.

It should be understood that it would also be acceptable to combine theembodiments and variant embodiments described above in various ways.

While various embodiments and variant embodiments have been explained inthe above description, the present invention is not to be considered asbeing limited to the details thereof. Other forms that are considered tocome within the scope of the technical concept of the present inventionare also to be understood as being included within the range of thepresent invention.

Image capturing devices, image processing devices, and display devicessuch as described below are also included in the embodiments and variantembodiments described above.

(1) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that performscorrection upon the second signal outputted from the second imagecapturing region, for employment in interpolation of the first signaloutputted from the first image capturing region; and a generation unitthat generates an image of the photographic subject that has beencaptured by the first image capturing region by employing a signalgenerated by interpolating the first signal by the second signal ascorrected by the correction unit.

(2) The image capturing device as described in (1) above, wherein, ascorrection for employment in interpolation of the first signal outputtedfrom the first image capturing region, the correction unit performscorrection upon the second signal outputted from the second imagecapturing region on the basis of the image capture condition for thefirst image capturing region and the image capture condition for thesecond image capturing region.

(3) The image capturing device as described in (2) above, wherein, ascorrection for employment in interpolation of the first signal outputtedfrom the first image capturing region, the correction unit performscorrection upon the second signal outputted from the second imagecapturing region on the basis of a difference between the image capturecondition for the first image capturing region and the image capturecondition for the second image capturing region.

(4) The image capturing device as described in (3) above, wherein: theimage capture condition for the first image capturing region and theimage capture condition for the second image capturing region areexposure conditions; and the correction unit performs correction uponthe second signal outputted from the second image capturing region onthe basis of a difference between the exposure condition for the firstimage capturing region and the exposure condition for the second imagecapturing region.

(5) The image capturing device as described in (4) above, wherein: theexposure condition for the first image capturing region and the exposurecondition for the second image capturing region are charge accumulationtimes of light reception units of the image capturing portions; and thecorrection unit performs correction upon the second signal outputtedfrom the second image capturing region on the basis of the ratio betweenthe charge accumulation time for the first image capturing region andthe charge accumulation time for the second image capturing region.

(6) The image capturing device as described in (4) above, wherein: theexposure condition for the first image capturing region and the exposurecondition for the second image capturing region are image capturesensitivities of the image capturing regions; and the correction unitperforms correction upon the second signal outputted from the secondimage capturing region on the basis of the logarithmic ratio of theimage capture sensitivity for the first image capturing region and theimage capture sensitivity for the second image capturing region.

(7) The image capturing device as described in (1) above, wherein, ascorrection for employment in interpolation of the first signal outputtedfrom the first image capturing region, the correction unit performscorrection upon the second signal outputted from the second imagecapturing region on the basis of the image capture condition for thefirst image capturing region or the image capture condition for thesecond image capturing region.

(8) The image capturing device as described in (1) above, wherein: thecorrection unit corrects the first signal outputted from the first imagecapturing region, and performs correction upon the second signaloutputted from the second image capturing region for employment ininterpolation of the corrected first signal; and the generation unitgenerates an image of the photographic subject captured by the firstimage capturing region by employing a signal generated by interpolatingthe corrected first signal by the second signal as corrected by thecorrection unit.

(9) The image capturing device as described in any one of (1) through(8) above, wherein: the first image capturing region photoelectricallyconverts light of a first color among the light from the photographicsubject and outputs the first signal; and the second image capturingregion photoelectrically converts light, among the light from thephotographic subject, of a second color that is different from the firstcolor, and outputs the second signal.

(10) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that performscorrection upon the second signal outputted from the second imagecapturing region, for employment in interpolation of a pixel in thefirst image capturing region that outputs the first signal; and ageneration unit that generates an image of the photographic subject thathas been captured by the first image capturing region by employing asignal generated by interpolating the pixel that outputs the firstsignal with the second signal corrected by the correction unit.

(11) The image capturing device as described in (10) above, wherein: apixel of the first image capturing region that outputs the first signalphotoelectrically converts light of a first color, among the light fromthe photographic subject, and outputs the first signal; and the secondimage capturing region photoelectrically converts light, among the lightfrom the photographic subject, of a second color that is different fromthe first color, and outputs the second signal.

(12) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that performscorrection upon the second signal outputted from the second imagecapturing region, for reducing noise included in the first signaloutputted from the first image capturing region; and a generation unitthat reduces noise included in the first signal by employing the secondsignal as corrected by the correction unit, and generates an image ofthe photographic subject that has been captured by the first imagecapturing region.

(13) The image capturing device as described in (12) above, wherein, ascorrection for reducing noise included in the first signal outputtedfrom the first image capturing region, the correction unit performscorrection upon the second signal outputted from the second imagecapturing region on the basis of the image capture condition for thefirst image capturing region or the image capture condition for thesecond image capturing region.

(14) The image capturing device as described in (12) above, wherein, ascorrection for reducing noise included in the first signal outputtedfrom the first image capturing region, the correction unit performscorrection upon the second signal outputted from the second imagecapturing region on the basis of the image capture condition for thefirst image capturing region and the image capture condition for thesecond image capturing region.

(15) The image capturing device as described in (14) above, wherein, ascorrection for employment for interpolation of the first signaloutputted from the first image capturing region, the correction unitperforms correction upon the second signal outputted from the secondimage capturing region on the basis of the difference between the imagecapture condition for the first image capturing region and the imagecapture condition for the second image capturing region.

(16) The image capturing device as described in (15) above, wherein: theimage capture condition for the first image capturing region and theimage capture condition for the second image capturing region areexposure conditions; and the correction unit performs correction uponthe second signal outputted from the second image capturing region onthe basis of the difference between the exposure condition for the firstimage capturing region and the exposure condition for the second imagecapturing region.

(17) The image capturing device as described in (16) above, wherein: theexposure condition for the first image capturing region and the exposurecondition for the second image capturing region are charge accumulationtimes of light reception units of the image capturing regions; and thecorrection unit performs correction upon the second signal outputtedfrom the second image capturing region on the basis of the ratio betweenthe charge accumulation time for the first image capturing region andthe charge accumulation time for the second image capturing region.

(18) The image capturing device as described in (16) above, wherein: theexposure condition for the first image capturing region and the exposurecondition for the second image capturing region are image capturingsensitivities of the image capturing regions; and the correction unitperforms correction upon the second signal outputted from the secondimage capturing region on the basis of the logarithmic ratio of theimage capture sensitivity for the first image capturing region and theimage capture sensitivity for the second image capturing region.

(19) The image capturing device as described in (15) above, wherein: thecorrection unit corrects the first signal outputted from the first imagecapturing region, and performs correction upon the second signaloutputted from the second image capturing region for reducing noiseincluded in the first signal after correction; and the generation unitgenerates an image of the photographic subject captured by the firstimage capturing region by reducing noise included in the corrected firstsignal by the second signal as corrected by the correction unit.

(20) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that performscorrection upon the second signal outputted from the second imagecapturing region, for employment in image processing of the first signaloutputted from the first image capturing region; and a generation unitthat generates an image of the photographic subject that has beencaptured by the first image capturing region by image processing thefirst signal by employing the second signal as corrected by thecorrection unit.

(21) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region and an image capturecondition for the second image capturing region; a processing unit that,as processing to interpolate the first signal by employing the secondsignal outputted from the second image capturing region, performsdifferent processing by using the image capture condition set for thesecond image capturing region by the setting unit; and a generation unitthat generates an image of the photographic subject that has beencaptured by the first image capturing region, by employing a signalgenerated by interpolating the first signal by the processing unit.

(22) The image capturing device as described in (21) above, wherein, asprocessing to interpolate the first signal by employing the secondsignal outputted from the second image capturing region, the processingunit performs different processing when a first image capture conditionis set by the setting unit for the first image capturing region and thesecond image capturing region, and when the first image capturecondition is set for the first image capturing region and a second imagecapture condition that is different from the first image capturecondition is set for the second image capturing region.

(23) The image capturing device as described in (22) above, wherein: theprocessing unit comprises a correction unit that, when the first imagecapture condition is set by the setting unit for the first imagecapturing region, and a second image capture condition that is differentfrom the first image capture condition is set for the second imagecapturing region, performs correction upon the second signal outputtedfrom the second image capturing region for employment in interpolationof the first signal outputted from the first image capturing region; andthe generation unit generates an image of the photographic subjectcaptured by the first image capturing region, by employing a signalgenerated by interpolating the first signal according to the secondsignal as corrected by the correction unit, as a signal resulted fromthe interpolation of the first signal by the processing unit.

(24) The image capturing device as described in (22) above, wherein,when the first image capture condition has been set by the setting unitfor the first image capturing region and for the second image capturingregion, the generation unit generates an image of the photographicsubject captured by the first image capturing region, by employing asignal generated by interpolating the first signal according to thesecond signal, as a signal resulted from the interpolation of the firstsignal by the processing unit.

(25) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that corrects the firstsignal outputted from the first image capturing region, and performscorrection upon the second signal outputted from the second imagecapturing region, for employment in interpolation of the corrected firstsignal; and a generation unit that generates an image of thephotographic subject that has been captured by the first image capturingregion by employing a signal generated by interpolating the first signalas corrected by the correction unit, according to the second signal ascorrected by the correction unit.

(26) The image capturing device as described in (25) above, wherein thecorrection unit performs correction upon the first signal on the basisof a set value that is different from a set value for the image capturecondition for the first image capturing region and from a set value forthe image capture condition for the second image capturing region, andthe set value for the image capture condition for the first imagecapturing region, and performs correction upon the second signal foremployment in interpolation of the corrected first signal, on the basisof the set value for the image capture condition for the second imagecapturing region, and the different set value.

(27) The image capturing device as described in (25) above, wherein thecorrection unit corrects the first signal on the basis of the differencebetween the set value for the image capture condition for the firstimage capturing region and the different set value, and corrects thesecond signal on the basis of the difference between the set value forthe image capture condition for the second image capturing region andthe different set value.

(28) The image capturing device as described in (27) above, wherein thecorrection unit corrects the first signal on the basis of the ratio ofthe different set value to the set value for the image capture conditionfor the first image capturing region, and corrects the second signal onthe basis of the ratio of the different set value to the set value forthe image capture condition for the second image capturing region.

(29) An image capturing device, comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a setting unit that sets an image capturecondition for the first image capturing region to an image capturecondition that is different from an image capture condition for thesecond image capturing region; a correction unit that corrects the firstsignal outputted from the first image capturing region, and performscorrection upon the second signal outputted from the second imagecapturing region to be employed in interpolation of a pixel that outputsthe corrected first signal; and a generation unit that generates animage of the photographic subject that has been captured by the firstimage capturing region by employing a signal generated by interpolatingthe pixel that outputs the first signal, that is corrected according tothe second signal as corrected by the correction unit.

(30) The image capturing device as described in (29) above, wherein: apixel of the first image capturing region that outputs the first signalphotoelectrically converts light of a first color, among the light fromthe photographic subject, and outputs the first signal; and the secondimage capturing region photoelectrically converts light of a secondcolor, among the light from the photographic subject, that is differentfrom the first color, and outputs the second signal.

(31) An image processing device, comprising: a correction unit that, foremployment in interpolation of a first signal outputted from a firstimage capturing region of an image capturing element that captures animage of a photographic subject, performs correction upon a secondsignal outputted from a second image capturing region of the imagecapturing element for which an image capture condition is different froman image capture condition for the first image capturing region; and ageneration unit that generates an image of the photographic subject thathas been captured by the first image capturing region by employing asignal generated by interpolating the first signal according to thesecond signal as corrected by the correction unit.

(32) An image processing device, comprising: a correction unit that, inorder to reduce noise included in a first signal outputted from a firstimage capturing region of an image capturing element that captures animage of a photographic subject, performs correction upon a secondsignal outputted from a second image capturing region of the imagecapturing element for which an image capture condition is different froman image capture condition for the first image capturing region; and ageneration unit that generates an image of the photographic subject thathas been captured by the first image capturing region, in which noiseincluded in the first signal is reduced by employing the second signalas corrected by the correction unit.

(33) A display device, comprising: a correction unit that, foremployment in interpolation of a first signal outputted from a firstimage capturing region of an image capturing element that captures animage of a photographic subject, performs correction upon a secondsignal outputted from a second image capturing region of the imagecapturing element for which an image capture condition is different froman image capture condition for the first image capturing region; and adisplay unit that displays an image of the photographic subject that hasbeen captured by the first image capturing region, generated byemploying a signal generated by interpolating the first signal accordingto the second signal as corrected by the correction unit.

(34) A display device, comprising: a correction unit that, for reducingnoise included in a first signal outputted from a first image capturingregion of an image capturing element that captures an image of aphotographic subject, performs correction upon a second signal outputtedfrom a second image capturing region of the image capturing element forwhich an image capture condition is different from an image capturecondition for the first image capturing region; and a display unit thatdisplays an image of the photographic subject that has been captured bythe first image capturing region, generated by reducing noise includedin the first signal by employing the second signal as corrected by thecorrection unit.

(35) An image capturing device, comprising: an image capturing unithaving a first region that captures incident light, and a second region,different from the first region, that captures incident light; a settingunit that sets an image capture condition for the first region so as tobe different from an image capture condition for the second region; acorrection unit that corrects first image data generated by capturinglight incident upon the first region under the image capture conditionset by the setting unit, on the basis of the image capture condition forthe second region set by the setting unit; and a generation unit thatgenerates an image according to the first image data as corrected by thecorrection unit, and second image data generated by capturing lightincident upon the second region.

(36) An image capturing device, comprising: an image capturing unithaving a first region that captures incident light, and a second region,different from the first region, that captures incident light; a settingunit that sets an image capture condition for the first region so as tobe different from an image capture condition for the second region; acorrection unit that corrects first image data generated by capturinglight incident upon the first region under the image capture conditionset by the setting unit on the basis of the image capture condition forthe second region set by the setting unit, and that corrects secondimage data generated by capturing light incident upon the second regionunder the image capture condition set by the setting unit on the basisof the image capture condition for the first region set by the settingunit; and a generation unit that generates an image according to thefirst image data as corrected by the correction unit, and the secondimage data as corrected by the correction unit.

(37) An image processing device, comprising: a setting unit that sets animage capture condition for a first region of an image capturing unit soas to be different from an image capture condition for a second regionof the image capturing unit; a correction unit that corrects first imagedata generated by capturing light incident upon the first region underthe image capture condition set by the setting unit, on the basis of theimage capture condition for the second region set by the setting unit;and a generation unit that generates an image from the first image dataas corrected by the correction unit, and second image data generated bycapturing light incident upon the second region.

(38) An image processing device, comprising: a setting unit that sets animage capture condition for a first region of an image capturing unit soas to be different from an image capture condition for a second regionof the image capturing unit; a correction unit that corrects first imagedata generated by capturing light incident upon the first region underthe image capture condition set by the setting unit on the basis of theimage capture condition for the second region set by the setting unit,and corrects second image data generated by capturing light incidentupon the second region under the image capture condition set by thesetting unit on the basis of the image capture condition for the firstregion set by the setting unit; and a generation unit that generates animage from the first image data as corrected by the correction unit, andsecond image data generated by capturing light incident upon the secondregion.

(39) A display device, comprising: a setting unit that sets an imagecapture condition for a first region of an image capturing unit so as tobe different from an image capture condition for a second region of theimage capturing unit; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit, on the basis of theimage capture condition for the second region set by the setting unit;and a display unit that displays an image generated from the first imagedata as corrected by the correction unit, and second image datagenerated by capturing light incident upon the second region.

(40) A display device, comprising: a setting unit that sets an imagecapture condition for a first region of an image capturing unit so as tobe different from an image capture condition for a second region of theimage capturing unit; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit on the basis of theimage capture condition for the second region set by the setting unit,and corrects second image data generated by capturing light incidentupon the second region under the image capture condition set by thesetting unit on the basis of the image capture condition for the firstregion set by the setting unit; and a display unit that displays animage generated from the first image data as corrected by the correctionunit, and second image data generated by capturing light incident uponthe second region.

Furthermore, image capturing devices, image processing devices, anddisplay devices such as described below are also included in theembodiments and variant embodiments described above.

(1) An image capturing device, comprising: an image capturing unithaving a first region that captures incident light, and a second region,different from the first region, that captures incident light; a settingunit that sets an image capture condition for the first region so as tobe different from an image capture condition for the second region; acorrection unit that corrects first image data generated by capturinglight incident upon the first region under the image capture conditionset by the setting unit, on the basis of the image capture condition forthe second region set by the setting unit; and a generation unit thatgenerates an image from the first image data as corrected by thecorrection unit, and second image data generated by capturing lightincident upon the second region.

(2) The image capturing device as described in (1) above, wherein thecorrection unit corrects the first image data so that the differencebetween the value of the first image data and the value of the secondimage data becomes smaller.

(3) The image capturing device as described in (1) or (2) above, whereinthe correction unit corrects the first image data on the basis of thedifference between the image capture condition for the first region andthe image capture condition for the second region.

(4) The image capturing device as described in any one of (1) through(3) above, wherein the generation unit performs image processing uponthe second image data by employing the first image data as corrected bythe correction unit.

(5) The image capturing device as described in any one of (1) through(4) above, wherein the generation unit changes the value of the secondimage data by employing the value of the first image data as correctedby the correction unit.

(6) The image capturing device as described in any one of (1) through(5) above, wherein the generation unit replaces the value of the secondimage data with the value of the first image data as corrected by thecorrection unit.

(7) The image capturing device as described in any one of (1) through(6) above, wherein the generation unit generates third image data fromthe second image data and the first image data as corrected by thecorrection unit.

(8) The image capturing device as described in any one of (1) through(7) above, wherein the setting unit, as the image capture condition forthe first region of the image capturing unit, sets an image capturesensitivity to a first image capture sensitivity, and, as the imagecapture condition for the second region of the image capturing unit,sets an image capture sensitivity to a second image capture sensitivitythat is different from the first image capture sensitivity, and thecorrection unit corrects the first image data generated by image captureunder the image capture condition set by the setting unit, on the basisof the difference between the first image capture sensitivity and thesecond image capture sensitivity.

(9) An image capturing device, comprising: an image capturing unithaving a first region that captures incident light, and a second region,different from the first region, that captures incident light; a settingunit that sets an image capture condition for the first region so as tobe different from an image capture condition for the second region; acorrection unit that corrects first image data generated by capturinglight incident upon the first region under the image capture conditionset by the setting unit on the basis of the image capture condition forthe second region set by the setting unit, and that corrects secondimage data generated by capturing light incident upon the second regionunder the image capture condition set by the setting unit on the basisof the image capture condition for the first region set by the settingunit; and a generation unit that generates an image from the first imagedata as corrected by the correction unit, and the second image data ascorrected by the correction unit.

(10) The image capturing device as described in (9) above, wherein thecorrection unit corrects the first image data and the second image dataso that the difference between the value of the first image data and thevalue of the second image data becomes smaller.

(11) The image capturing device as described in (10) above, wherein thecorrection unit performs correction so as to cause the value of thefirst image data to increase and so as to cause the value of the secondimage data to decrease.

(12) The image capturing device as described in any one of (9) through(11) above, wherein the correction unit corrects the first image data onthe basis of the image capture condition for the first region and theimage capture condition for the second region set by the setting unit,and corrects the second image data on the basis of the image capturecondition for the first region and the image capture condition for thesecond region.

(13) The image capturing device as described in (12) above, wherein thecorrection unit corrects the first image data and the second image dataon the basis of the difference between the image capture condition forthe first region and the image capture condition for the second region.

(14) The image capturing device as described in any one of (9) through(13) above, wherein the generation unit performs image processing uponthe second image data as corrected by the correction unit, by employingthe first image data as corrected by the correction unit.

(15) The image capturing device as described in any one of (9) through(14) above, wherein the generation unit corrects the image that has beengenerated.

(16) An image processing device, comprising: a setting unit that sets animage capture condition for a first region of an image capturing unit soas to be different from an image capture condition for a second regionof the image capturing unit; a correction unit that corrects first imagedata generated by capturing light incident upon the first region underthe image capture condition set by the setting unit, on the basis of theimage capture condition for the second region set by the setting unit;and a generation unit that generates an image from the first image dataas corrected by the correction unit, and second image data generated bycapturing light incident upon the second region.

(17) The image processing device as described in (16) above, wherein thecorrection unit corrects the first image data so that the differencebetween the value of the first image data and the value of the secondimage data becomes smaller.

(18) The image processing device as described in (16) or (17) above,wherein the correction unit corrects the first image data on the basisof the difference between the image capture condition for the firstregion and the image capture condition for the second region.

(19) An image processing device, comprising: a setting unit that sets animage capture condition for a first region of an image capturing unit soas to be different from an image capture condition for a second regionof the image capturing unit; a correction unit that corrects first imagedata generated by capturing light incident upon the first region underthe image capture condition set by the setting unit on the basis of theimage capture condition for the second region set by the setting unit,and corrects second image data generated by capturing light incidentupon the second region under the image capture condition set by thesetting unit on the basis of the image capture condition for the firstregion set by the setting unit; and a generation unit that generates animage from the first image data as corrected by the correction unit, andsecond image data generated by capturing light incident upon the secondregion.

(20) The image processing device as described in (19) above, wherein thecorrection unit corrects the first image data and the second image dataso that the difference between the value of the first image data and thevalue of the second image data becomes smaller.

(21) The image processing device as described in (19) or (20) above,wherein the correction unit corrects the first image data and the secondimage data on the basis of the difference between the image capturecondition for the first region and the image capture condition for thesecond region.

(22) A display device, comprising: a setting unit that sets an imagecapture condition for a first region of an image capturing unit so as tobe different from an image capture condition for a second region of theimage capturing unit; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit, on the basis of theimage capture condition for the second region set by the setting unit;and a display unit that displays an image generated from the first imagedata as corrected by the correction unit, and second image datagenerated by capturing light incident upon the second region.

(23) The display device as described in (22) above, wherein thecorrection unit corrects the first image data so that the differencebetween the value of the first image data and the value of the secondimage data becomes smaller.

(24) The display device as described in (22) or (23) above, wherein thecorrection unit corrects the first image data on the basis of thedifference between the image capture condition for the first region andthe image capture condition for the second region.

(25) A display device, comprising: a setting unit that sets an imagecapture condition for a first region of an image capturing unit so as tobe different from an image capture condition for a second region of theimage capturing unit; a correction unit that corrects first image datagenerated by capturing light incident upon the first region under theimage capture condition set by the setting unit on the basis of theimage capture condition for the second region set by the setting unit,and corrects second image data generated by capturing light incidentupon the second region under the image capture condition set by thesetting unit on the basis of the image capture condition for the firstregion set by the setting unit; and a display unit that displays animage generated from the first image data as corrected by the correctionunit, and second image data generated by capturing light incident uponthe second region.

(26) The display device as described in (25) above, wherein thecorrection unit corrects the first image data and the second image dataso that the difference between the value of the first image data and thevalue of the second image data becomes smaller.

(27) The display device as described in (25) or (26) above, wherein thecorrection unit corrects the first image data and the second image dataon the basis of the difference between the image capture condition forthe first region and the image capture condition for the second region.

(28) The image capturing device as described in any one of (1) through(15) above, wherein: the correction unit comprises a first correctionunit that corrects the first image data that has been generated bycapturing light incident upon the first region under the image capturecondition set by the setting unit, and a second correction unit thatcorrects the second image data that has been generated by capturinglight incident upon the second region under the image capture conditionset by the setting unit; the first correction unit corrects the firstimage data on the basis of the image capture condition for the secondregion; and the generation unit generates an image from the first imagedata as corrected by the first correction unit and the second imagedata.

(29) The image capturing device as described in (28) above, wherein thefirst correction unit corrects the first image data so that thedifference between the value of the first image data and the value ofthe second image data becomes smaller.

(30) The image capturing device as described in (28) or (29) above,wherein the first correction unit corrects the first image data on thebasis of the difference between the image capture condition for thefirst region and the image capture condition for the second region.

(31) The image capturing device as described in any one of (28) through(30) above, wherein the second correction unit corrects the second imagedata on the basis of the image capture condition for the first region,and the generation unit generates an image from the first image data ascorrected by the first correction unit and the second image data ascorrected by the second correction unit.

(32) The image capturing device as described in any one of (28) through(31) above, wherein the second correction unit corrects the second imagedata according to the first image data, and the generation unitgenerates an image from the first image data as corrected by the firstcorrection unit and the second image data as corrected by the secondcorrection unit.

(33) The image capturing device as described in any one of (28) through(32) above, wherein the image capturing unit is provided at a firstsemiconductor substrate, and the first correction unit and the secondcorrection unit are provided at a second semiconductor substrate.

(34) The image capturing device as described in (33) above, wherein thefirst semiconductor substrate and the second semiconductor substrate arelaminated together.

The content of the disclosure of the following base application, uponwhich priority is claimed, is hereby incorporated herein by reference:

-   Japanese Patent Application No. 2015-194612 (filed on 30 Sep. 2015).

REFERENCE SIGNS LIST

-   -   1, 1C . . . cameras    -   1B . . . image capturing system    -   32 . . . image capturing unit    -   32 a, 100 . . . imaging elements    -   33 . . . image processing unit    -   33 a, 321 . . . input unit    -   33 b, 322 . . . correction units    -   33 c, 323 . . . generation units    -   34 . . . control unit    -   34 a . . . object detection unit    -   34 b . . . setting unit    -   34 c . . . image capturing control unit    -   34 d . . . AF calculation unit    -   35 . . . display unit    -   90 . . . predetermined range    -   1001 . . . image capturing device    -   1002 . . . display device    -   P . . . pixel for attention

1. An image capturing device comprising: an image capturing elementhaving a first image capturing region that captures an image of aphotographic subject and outputs a first signal, and a second imagecapturing region that captures an image of the photographic subject andoutputs a second signal; a central processor configured to enablesetting of a different image capture condition for the first imagecapturing region and the second image capturing region; and an imageprocessor configured to (i) perform correction upon the second signaloutputted from the second image capturing region based on the imagecapture condition for the first image capturing region and the imagecapture condition for the second image capture region, and (ii) use thesecond signal upon which the correction has been performed to performinterpolation of the first signal outputted from the first imagecapturing region.